FreeSWITCH – IVRs and Fun Stuff

In my previous post I covered how to setup FreeSWITCH behind a PAN firewall, connect with a gateway (the ITSP) and configure a very basic default and public dialplans to get simple inbound and bound calls working.

For the best reading I suggest this book. 🙂

FreeSwitch

IVRs

The path when working with IVRs is:

/etc/freeswitch/ivr_menus/

inside we can see the following:

root@FreeSwitch:~# ls /etc/freeswitch/ivr_menus/
demo_ivr.xml  my_ivr.xml  new_demo_ivr.xml

not a whole lot. First thing I wanted to figure out was how calling ext 5000 would lead to this IVR.

So looking at the default internal dial-plan, we can see it

nano /etc/freeswitch/dialplan/default.xml
    <!-- a sample IVR  -->
    <extension name="ivr_demo">
      <condition field="destination_number" expression="^5000$">
        <action application="answer"/>
        <action application="sleep" data="2000"/>
        <action application="ivr" data="demo_ivr"/>
      </condition>
    </extension>

Looks like it’s the application IVR data=demo_ivr. Quick way to find out let’s rename the file and see if the dialplan still works..ok, it’s not the filename, but the name value in the xml files located within the folder specified above.

Now as you can see I managed to get the screaming monkeys to work by simply recording a stream of screaming monkeys and exporting it with audacity in compressed ULaw wav format, uploaded it to freeSWITCH via WinSCP. then changed the action type and param value.

Now if you get into the nitty gritty, you’ll notice the default IVR uses phrases which are pieced together pieces of smaller recordings. These you may notice by default are also relatively referenced, instead of fully “ivr/ivr-that-was-invalid-entry.wav” which you may notice from searching exists only in the language folders of the sounds…

for the time being I won’t get into making that custom of an IVR, instead start off simple. I’m gonna create an audio recording of my options (1 for sales, 2 for work, 3 for support, 4 for other). Then used WinSCP to copy to the Freeswitch server, then copied to location.

root@FreeSwitch:/etc/freeswitch/ivr_menus# cp /home/zewwy/ZewwyCA.wav /usr/share/freeswitch/sounds/
root@FreeSwitch:/etc/freeswitch/ivr_menus# ls /usr/share/freeswitch/sounds/
en  es  fr  music  pt  ru  ScreamingMonkey.wav  ZewwyCA.wav

ok now, let’s make the IVR do stuff with these options….

first we’ll set the caller ID name and number in vars.xml so we can use the default variables in our directories, and hopefully for outbound calls.

So with my recording in place, i created my IVR as follows:

nano /etc/freeswitch/ivr_menus/zewwy_ivr.xml

then created a new extension to reach in my default dialplan

nano /etc/freeswitch/dialplan/default.xml

Now I just need to change my public dialplan to call this extension instead of my SPAPhone directly.

Change 1002 to 5006. That’s it. 😀 now got a IVR.

Time of Day Call Routing

I used FreeSwitch’s own post on this as a reference.

For me, I created option one in the options for “Sales”. I don’t want to be bothered about items for sales when I’m sleeping, or off work. Currently as above you can see calls are going straight to my voice mail. Well let’s change that…

<!-- My Cell -->
<extension name="mycell">
  <condition field="destination_number" expression="^mycell$">
    <action application="bridge" data="sofia/gateway/${default_gateway}/1#######"/>
  </condition>
</extension>

First, let’s create a new extension for Sales; 2222: in default dialplan

<extension name="Sales-x2222">
  <condition field="destination_number" expression="^2222$">
    <action application="transfer" data="Sales"/>
  </condition>
</extension>

just below that:

<extension name="Sales" continue="true">
<condition field="destination_number" expression="^Sales$"/>
  <condition wday="2-6" hour="9-18">
    <action application="set" data="Sales_open=true"/>
    <action application="transfer" data="xfer-to-sales"/>    
    <anti-action application="set" data="Sales_open=false"/>
    <anti-action application="transfer" data="xfer-to-sales"/>
  </condition>
</extension>
<extension name="xfer-to-sales">
  <condition field="destination_number" expression="^xfer-to-sales$"/>
  <condition field="${Sales_open}" expression="^true$">
    <action application="transfer" data="mycell"/>
    <action application="answer"/>
    <action application="sleep" data="2000"/>
    <action application="voicemail" data="default ${domain_name} 1002"/>
    <anti-action application="voicemail" data="default ${domain_name} 1002"/>
  </condition>
</extension>

If you can’t tell what is happening here, we are creating a dial number named Sales-x2222 when you dial 2222. Then we define the sales work hours Weekdays from 9 till 6, which defines the normal action lines, in this case it sets a variable “Sales_open” to true, otherwise if not within this time, set the variable to false. In the third area we use this flag to either call mycell or leave a voicemail on ext 1002.

Now I simply changed the first line in my IVR instead of bridging a call to my cell, I send it to ext 2222 which will only call my cell during working hours. 🙂

*NOTE* I could have also simply done this directly under the Public based dialplan such as this:

<include>
	<expression name="public_did">
		<condition field="destination_number" expression="1#######$">
			<action application="answer"/>
			<condition wday="2-6" hour="9-18">
				<action application="transfer" data="callfwd">
				<anti-action application="ivr" data="zewwy_ivr">
			</condition>
		</condition>
	</expression>
	<expression name="callfwd">
		<condition field="destination_number" expression="^callfwd$">
			<action application="answer"/>
			<action application="speak" data="flite|rms|Calling someone in reguards to a item for sale. Hold please."/>
			<action application="set" data="effective_caller_id_name=SALE(${caller_id_name})"/>
			<action application="set" data="effective_caller_id_number=${caller_id_number}"/>
			<action application="bridge" data="sofia/gateway/${default_gateway}/1#######"/>
		</condition>
	</expression>
</include>

However I wanted my IVR to stay the same no matter when it was called. So I placed my time of day routing at the internal dial-plan on the specific use case/department. It was simply a more scalable example to use.

Blast Group

here’s an example of a 2 phone blast group via ext 511 using the default dial plan, blasting ext’s 1002 and 1003

this took me a bit…

Hunt Group

ext 512 dials Ext 1001, if not available call 1002 (ring only for 20 second), if not available ring ext 1003 (for 30 seconds), if no answer drop to 1002’s voice mailbox.

some Music on Hold would be nice while the call is being transferred…

    <extension name="a_blast_group">
      <condition field="destination_number" expression="^511$">
        <action application="set" data="ringback=${us-ring}"/>
        <action application="set" data="transfer_ringback=$${hold_music}"/>
        <action application="answer"/>
        <action application="sleep" data="2000"/>
        <action application="bridge" data="sofia/$${domain}/1002,sofia/$${domain}/1003"/>
      </condition>
    </extension>

    <extension name="a_hunt_group">
      <condition field="destination_number" expression="^512$">
        <action application="set" data="ringback=${us-ring}"/>
        <action application="set" data="transfer_ringback=$${hold_music}"/>
        <action application="answer"/>
        <action application="sleep" data="2000"/>
        <action application="set" data="continue_on_fail=true"/>
        <action application="set" data="call_timeout=20"/>
        <action application="bridge" data="sofia/$${domain}/1001|sofia/$${domain}/1002"/>
        <action application="set" data="call_timeout=30"/>
        <action application="bridge" data="sofia/$${domain}/1003"/>
        <action application="voicemail" data="default ${domain_name} 1002"/>
      </condition>
    </extension>

Hope someone found this guide useful. 🙂

FreeSWITCH

The Story

My buddy Troy did a presentation, I wanna try it out. So this is going to be a shit show… let’s go…

Sources: Specs

Minimum/Recommended System Requirements
32-bit OS (64-bit recommended)
512MB RAM (1GB recommended)
50MB of Disk Space
System requirements depend on your deployment needs. We recommend you plan for 50% duty cycle.

Install Source for Debian 10

Buddy Troys Presentation

Install Debian 10

So I’ll setup a VM with those nice minimum requirements, could def use the memory savings, most servers these days are redic.

LimbooooooooOOOOOOOOOooo! How low can you go?!

Alright let’s install Debian 10.

Install Source Info and Install Source Image I’ll use the netinstall image.

Mount image to VM… booot er up (I’m gonna try EFI instead of BIOS)

Nice, it booted, Install Graphical or Install, just install, we want to keep it CLI only as it has bare resource allocations.

set root password, create alternative user, guided use entire disk, or set however you like, or however you deploy your AC3 AWS nodes or whatever cloud based instance you might be using. Whatever floats your digital boats.

*Digitized Voice* All your base are belong to us…

ohhh boy…. anyway.

SSH and Standard system packages… this installer keeps going…

Wooo never thought I’d see the day… OK so now that we finally have a clean Debian server, we can move to the next step. 😀

FreeSwitch Install

From Source: “Debian 10 “Buster” is the reference platform for FreeSWITCH™ as of version 1.10

Dependencies are available from FreeSWITCH repository via the “apt-get build-dep freeswitch” command.”

ok let’s try that?

Not sure why that’s at the top of the document when it doesn’t work out of the box, let’s follow along with the “easy way” then…

apt-get update && apt-get install -y gnupg2 wget lsb-release

Moving on…

wget -O - https://files.freeswitch.org/repo/deb/debian-release/fsstretch-archive-keyring.asc | apt-key add -

# you may want to populate /etc/freeswitch at this point.
# if /etc/freeswitch does not exist, the standard vanilla configuration is deployed
apt-get update && apt-get install -y freeswitch-meta-all

Uhhh ok, I don’t have a config in mind per say so I guess I’ll use the predefined one without creating that directory or file… let’s go!

Off she goes 200+ already! That didn’t take too long. Let’s see if we can get into the freeswitch cli…

fs_cli -rRS

[ERROR] fs_cli.c:1565 main() Error Connecting [Socket Connection Error]

OK dokie then, let’s give er a good old reboot. After reboot, haza!

The Presentation in a Nut Shell

SIP (Session Initiation Protocol) -> Initiates the connection for the task
SDP (Session Description Protocol) -> Connection for what
RT(C)P (Real Time (Control) Protocol) -> RTP: Audio Packets RTCP: Metadata

Now slide 25 while very simple topology layout isn’t crazy it was the mentioning of alternative NAT tricks which kind of boggled my mind a bit. The other day I had issues with my Signal app using mobile data excessively even though I was on WiFi. Took me a little while to figure out but it was my firewall that was blocking the traffic and it appears Signal secretly uses any alternative networks on the device to establish the required connection. During the research for a solution, I found a PaloAltoNetworks thread on the issue

Creating a rule with the three main applications (Signal, SSL, STUN) allowing any service ports, and then turning off my mobile data. Still resulted in failed Signal calls. I have to open the rule up completely and even disable server response inspection. I had talked to my local PAN technical rep, I might just make a separate blog post about that entirely. Anyway just making note of that as a possible infrastructure to hurdle while I go through this endeavor…

Check out this Wiki Page on more details on STUN if you have the basic understanding of the difference between TCP and UDP the contents should be fairly easy to digest. However, I digress and move on.

Well it’s going to be harder than I thought to put all this info into a Nut Shell, so instead I’ll try to cover each piece of the puzzle one at a time. First thing on any server is to have a static IP (at least if your behind a NAT which is mentioned many times in his presentation, and I’ll discuss my setup and how that flies when we get to that step). For now let’s just set our internal static private IP address.

OK strange, coming back to this VM from yesterday I was still int the freeswitch CLI, yet typing /exit would bring up the same freeswitch CLI… so hard reboot… and… ok so the initial Debian install guide said to do fs_cli with some options. Read here for a PDF of details options truns out the -R is reconnect when disconnect, and /exit, /bye, /quit are all disconnects. So just use fs_cli without -R, and the /exit works without issue.

Set Host Static IP

Now with that annoyance out of the way, well use this Debian guide to set out IP as root.

nano /etc/network/interfaces

from:

auto eth0
allow-hotplug eth0
iface eth0 inet dhcp

to:

auto eth0
iface eth0 inet static
        address 192.0.2.22/24
        gateway 192.0.2.1

This leaves us so far with this very, very basic network diagram:

Super simple, but doesn’t cover the SIP connections coming in, and the following STUN and NAT traversal attempts, while most home routers may allow these connections the PA I’m behind, not so much. So we’ll again cover those details when we get to them.

*NOTE* I had to make this a Static IP NAT else I wouldn’t get Audio when doing direct sip calls not out to the regular PSTN. See this Post for more details. I also created the custom Apps and app override rules

Managing Users

Slide 33 starts off with:

sudo chmod g+w -R /etc/freeswitch

I’m starting to realize the slide out of context (being there for the presentation) is rather hard to follow along with, but since I’m doing everything as root for now (Since I didn’t add my standard user to the FreeSwitch group) I’ll just ignore this line for now.

  • cd /etc/freeswitch/directory/default
  • Remove everything but 1000.xml, 1001.xml
  • Edit for our user accounts
  • We can reload this configuration without restarting freeswitch

Ok dokie….

Neat! I had no idea you could do that! so do…

rm -v !("1000.xml"|"1001.xml")

That was so easy! 🙂 Since I’m again reading this out of context I’m not sure what edits were made to what files for the line “edit for our user accounts”. Looks like just the XML files we left behind, these appear to be template, as usual XML based, so knowing which fields to edit can be a bit trick to point out.

OK… a bit more details from the main source

That makes sense… so my final config:

Connecting a Phone

Before we can make any calls we are going to need a phone, now this can be almost any device, a laptop with softphone software , or can be a physical phone as long as it supports the SIP protocol. Lucky for me I have 2 different devices at my disposal for testing. Two older Cisco phones: 1) a Cisco SPA525g and 2) a Cisco WiFi 7925.

The desk phone I feel still looks nice and modern despite it’s age, the 7925 looks like an old brick Nokia and you can imagine the software is just as bad. So let’s see how we can get this into the mix. Lucky for me both support SIP.

Accordingly to the slides on slide 32 looks like we have to define the server listening address and port (which we will leave at the default 5060) that will be the unencrypted default port.

nano /etc/freeswitch/vars.xml

Oh yeah… that default password thing mentioned in slide 31, not sure why this would be clear text in a clear text xml file on a open config path, but… *Smiles n Nods* changing cleartext password.

Since I’m not making any changes at this time, we will just exit and cover applying config changes once we get there. I need to double check some sources to get my head around all this stuff right now, so please bare with me on this blog post, I’m literally learning everything as I go.

7925 – Settings locked (Press **# to unlock settings)
Setup all the WiFi settings then connected to my local network. Use this Cisco Doc for help configuring a Cisco 7925

This is were things kind of went sour. the Cisco 7925g is SCCP or Cisco protocol only (lame) so no SIP. literally Nothing. There maybe a way to use Skippy mod for FreeSwitch but well see about that in a future post.

As for the SPA525g It took me a good while of digging before I figured it out.

  1. Step IP
  2. Log into Web interface http://IP
  3. Click on “Admin Login”
  4. You should now see a SIP tab, leave it, click on Ext 1
  5. Fill in the Proxy Address: (IP of freeswitch)
    User ID: 1001
    Password: (AS set in XML)

I finally managed to get a successful registration after this (but all the soft buttons were lit up and displaying 1001)

So the phone doesn’t look nice (yet) and we only have one, with our 7925 out of the picture for now, I guess I may have to rely on a softphone after all. :S

To adjust the alternative buttons click the “phone” tab and set each line to disabled, or name them as alternative lines as Louis does in this youtube video.

Now I just need to setup user 1000 since the 7925 was so nice to not use SIP, like at all SCCP only, no thanks for now. So, in today’s more modern times, I’ll just use a softphone. I decided to play with linphone on my andriod phone.

Open it up and four options appeared create account, use linphone account, use sip account, or fetch remote configuration… Use SIP account.

username: 1000
password: AsSetInXLM
Domain: IPofFreeSwitch
Name: Optional

And it connected!

OK so now out setup looks like this:

Now we have the very basics to make out first call: 2 Users and 2 Registered phones. So from the SPA525g, I dialed 1000, and sure enough my linphone rang, picked it up and had my first self configured SIP call. It was the usual self mocking type comments back and forth. After hanging up there was some feeling of accomplishment. But no time to stop here… there’s more fun to be had!

Unfortunately I was unable to make a call from linphone to the SPA phone cause as others have mentioned in the comments for some reason it auto adding +1 in front of all numbers dialed and it won’t simply ring ext 1001.

Important Tid Bits

  • Log directory – /var/log/freeswitch
  • Configuration directory – /etc/freeswitch
  • Database directory – /var/lib/freeswitch/db
    • Hosts SQLite databases
    • SQLite is the default database, many are supported
  • Daemon is configured via systemd
 sudo systemctl start freeswitch [or] service freeswitch start|stop|status
  • Administration – make yourself part of the freeswitch group
sudo usermod -aG freeswitch useraccount

Invaluable tools for administration – fs_cli (included in freeswitch)

let’s also install sngrep (this will come in handy later)

sudo apt install sngrep

FreeSwitch Configs

  • /etc/freeswitch/freeswitch.xml
    • This is the “point of entry” for configuration
    • It includes /etc/freeswitch/vars.xml, and does fileglob-includes for other important bits
      • autoload_configs/*.xml
        • This is where module configurations live (e.g. database connectivity, SIP stack, more)
      • dialplan/*.xml
        • This is where dialplans live (e.g. how do you dial out, IVRs, etc)
      • directory/*.xml
        • This is where user provisioning lives by default
  • /etc/freeswitch/vars.xml
    • This is where the “preprocessor variables” and generally very important variables live
    • You can think of it as “settings that you can’t change at runtime”
    • Includes all your favourite hits, such as:
      • <X-PRE-PROCESS cmd=”set” data=”default_password=1234″/>
        • Change this ASAP!
      • <X-PRE-PROCESS cmd=”stun-set” data=”external_rtp_ip=stun:stun.freeswitch.org”/>
      • <X-PRE-PROCESS cmd=”stun-set” data=”external_sip_ip=stun:stun.freeswitch.org”/>

*The “stun” entries are for NAT traversal; if you’re not behind a NAT device, you can change these to “host:your.domain” or your IP address.

Since our FreeSwitch is behind a NAT as shown in the first topology picture, I left these fields defaulted.

  • /etc/freeswitch/vars.xml
    • Let’s change our domain:
      • <X-PRE-PROCESS cmd=”set” data=”domain=$${local_ip_v4}”/>
    • Other notable entries:
      • <X-PRE-PROCESS cmd=”set” data=”internal_sip_port=5060″/>
        • SIP phones will register to your server on this port
      • <X-PRE-PROCESS cmd=”set” data=”external_sip_port=5080″/>
        • Calls will come from your ITSP on this port

At this point in his slides he goes on about making an external call, while I do plan on getting to that, I needed a VoIP provider so I’m currently working on getting a VoIP provider setup. In the meantime…

Voice MaiL

I sure enough left a phone ringing, for a good amount a rings it seemed and I was automatically transferred to a user voice mail, amazingly no configuration was required.

The softphone (Linphone) also didn’t seem to have an indicator for such a thing and after a bit of da googling, I found you simply dial *98.

On the SPA525g first time pressing the mail icon will ask you to enter the voicemail number, which I entered incorrectly and had to find this guide to help me figure how to change it.

Setup -> User Preferences -> Call Preferences -> VoiceMail

Although I was able to listen to the message I found I would always get cut off at 30 seconds.

Nope Any call gets cut off after 30 seconds… I’m about to give up on this shit…

Troubleshooting, Yay!

I did manage to get a bit of help from my buddy Troy and a nice user on the FreeSwitch channel on IRC in #FreeNode

We used sngrep and realized that I was not getting a ACK message from the phone.

As you can see no ACK….

Cyrillax from IRC mentioned enabling advanced debug…

sofia loglevel all 9
sofia global siptrace on

This will output a lot to the screen, so if you need to backscroll and are using putty ensure you add plenty of backscroll lines the default is 200, and that is not enough..

Checking the debug logs we can see the contact info is not what we wanted, the phone is trying to connect to the FreeSwitch via the public IP address:

Now on the SPA525g we entered Advanced config area opening up additional configurations and told the phone to use the outbound proxy after defining it (with the IP address of the FreeSwitch) which worked and we had calls with the IVR last as long as required. I’m not sure if this will suffice when it comes times for outbound calls, but well cover that when we get there. lol I’ve been saying that a lot.

I still wasn’t sure if the additional proxy configs was the right solution to the problem, although it did resolve the problem and acks were sent from the phone directly back to freeswitch. However every softphone I setup even after setting freeswitch to the proxy IP it wouldn’t work and I’d see the SDP sent with the Public contact in the field every time…

no matter how I configured the FreeSwitch XML config files I couldn’t seem to get it to provide the contact of the private IP not the public one, which I kept reading and hearing that’s normally what you want. I couldn’t see these requests for traffic in my Monitor tab of the Palo Alto firewall, so I thought it was a dud or wasn’t happening, but decided to create a U-Turn NAT rule anyway.

after committing I finally got an ACK! hahah from the firewall itself, kind of as expected since it TCP based, in this case and required to completed TCP’s 3 way handshake.

The diagram looks like this now:

Now things work, except for some reason I can’t call the softphone from the spa525g. But the Softphone can ring the SPA525g just fine…. ughhh my ignorance is causing gremlins! OK everyone can call 5000, and voicemail, but noone can call the softphone @ 1000. I’ll figure this one out tomorrow.

I decided to see if this was the problem, and reverted the outbound proxy settings I had added to the SPA525g. and sure enough go multiple SDP with no ACK, this time it was cause they were attempting to negotiate via UDP not TCP has my rule above I created for TCP only… OK let’s duplicate the rule and also allow UDP. Since it now is using UDP for the SDP and I did not define that port in my UTurn NAT rule, I”ll create another one for UDP but without source NAT translation… so it’ll look like this:

The rule looks like this now:

and after committing we get an ACK from the phone directly, without configuring an outbound proxy setting on the phone. 😀

Sure enough, on the SPA525g, everything works, calls to the 5000 built in IVR, VoiceMail, the works. Now lets try the softphone again… nope….

OK well I’m not sure if it’s the VIA field causing me grief or the fact that calls being made when routing from the FreeSwitch keep saying from 1001@freeswitch instead of the users making the call 1000. I removed the CIDR from users 1001.xml and copied it, changed the password, updated the config with:

fs_cli -x "reloadxml"

And oddly enough I was finally able to call the softphone on ext 1000. There was a long delay before the ringing started but it worked this time?! like what?

Sure enough I can call both ways now, but when I call 1001 from the softphone (1000) it rings right away, if I call 1000 from the SPAphone (1001) there’s a delay before the ringing starts. I’m not sure if this is some limitation of the app I’m using. I also have no idea how the heck making that change made the calls start working…

after creating two new users (copied 1001.xml and changed all 1001 to 1002 and 1003 respectively). Now calls going both ways are instant and all phones soft and SPA are working 100%.

I stand corrected… calling ext 5000 give me now a busy signal… this is starting to really annoy me…

Oh wait… right I changed 1001.xml with random 1005 numbers….

once I reverted this back to default as pictured at the top of this blog post, ext 5000 started working again… Not sure why this is but I guess it might be time to check out the dial plans?

30 Second Cut Off

Check to make sure the FreeSwitch Server is getting the required ACK. See Above for example.

Call Connects but No Audio with Direct SIP Routing

Check your internet connection NAT rule for the FreeSwitch server, ensure it is a Static NAT, not Dynamic IP and Port.

10 Second Delay In Call Answer

I searched this one up the other day, and I’ve heard it could be DNS (check you /etc/resolve.conf) mine was good. Heard it was due to STUN people set there STUN servers blank, this however will have consequences on the SDP contact information, so I wouldn’t recommend this, but it has been mentioned. In my case it was all of a sudden deliberate sleep execution due to not having changed the default password in vars.xml.

So yeah…. make sure you change the default password. then reloadxml in fs_cli.

Dial Plans and Phone Numbers

Different Dial Plan Directories

You may have noticed we have (used for internal phones)

 /etc/freeswitch/dialplan/default/

as well as (used by les.net dialing in)

 /etc/freeswitch/dialplan/public/

OK…. now we finally got past all the lower layer technical hurdles we can finally get to configuring the application itself. However we need to … collaborate with external sources. Now for me I’m lucky and have a local VoIP provider that is small in size but very technically aware, and much like Troy’s slide I use the same provider. Les.net

I tried to setup an account with them anonymously but that didn’t work as I had to call in as my account got suspended with fake info… Whomp wommmm womomomo.

So after I got my account verified, clicking on Order DID, pick the area, the area-code and any other information and the order details pop up (slide 37):

hahahaha, it’s cheaper for me to order a number for Fargo, ND then it is for me to order a local Winnipeg number… hahahah ahhhh… btw I am not using that number, I’ll still with the free DID for now, anyway…

The point is now we should have the basics in place to get FreeSwitch server registered with an external VoIP provider so we can make calls to the, sweet, sweet, candy… I mean public phone system. Sooo we are working on this:

As you can see the SIP/SDP/RTC/RTCP arrow is both ways, so if the les.net proxy send UDP based packets at me, the Palo Alto Firewalls will not know what to do with them, and drop em like they’re hot, drop em like they’re hot… When the Bi…. whoops going off track anyway, let’s create some rules to allow connection from our Internet telephony service provider (ITSP).

These details should be provided to you by your ITSP.

I thought about it a bit and did not create and open bi-directional NAT rule cause I’m sure my ITSP doesn’t want DNS and alternative requests from my freeswitch, so instead I created an open one way NAT rule that says anything from LesNet SIP proxy’s send it to my Freeswitch, in hopes those proxy’s are also setup to send only what they need to the right place. I still need a security rule though to make this work. So again I’ll leave it open, monitor the traffic and restrict the application or service ports accordingly.

Now that we got the firewall out of the way let’s go configure the FreeSwitch server.

/etc/freeswitch/sip_profiles/external/

This directory is for integrating with upstream providers. You can have multiple ITSP gateways. These handles incoming SIP traffic on port 5080
(Which we have our NAT and SEC rule so this should be good to go now)
Example: you could register a DID for multiple provinces, and have each trunk as it’s own gateway.
/etc/freeswitch/sip_profiles/internal.xml
This configures your internal profile (port 5060) for accepting connections from SIP phones (Which we already went through the nitty gritty above)

Let’s create a file in this directory, lesnet.xml

cp /etc/freeswitch/sip_profiles/external/example.xml /etc/freeswitch/sip_profiles/external/lesnet.xml

Use the username, password, and proxy provided by lesnet’s login page(To do this on LesNet have to create a new SIP Peer / Trunk, then click the edit button on it, this will present the required details to enter into the xml file.)

New profiles can be loaded at runtime

fs_cli
sofia profile external rescan
sofia status gateway proxy.sip.les.net

Note – “sofia” is the name of the SIP stack used by freeswitch.

WOW! it worked!

Since this was a connection from freeswitch to lesnet I didn’t see it hit my newly created rule instead it used my default home network outbound rule which was allowed.

My excitement was again short lived as I hit another road block (story of my life). Turns out I kept seeing repeated Registrations and 401 responses. I wasn’t sure of this and made a change to my external gateway…

nano /etc/freeswitch/sip_profiles/external/lesnet.xml

sofia profile external restart reloadxml

So to get out bound to even show up on the lesnet side some changes were required.

nano /etc/freeswitch/vars.xml
 <X-PRE-PROCESS cmd="set" data="default_provider=proxy3.sip.les.net"/>

then again some reloadxml

fs_cli -x "reloadxml"

now when we make calls it’s bust but at least they show on the call logs on the ITSP portal.

Incoming Calls

Now for incoming calls, after you verify a stable connection with the ITSP Gateway/proxy, and see it their online portal, you may have to map a number to a DID Peer/Trunk, In this case I saw my registered FreeSwitch as SIP Peer 79908, then under “Your DIDs” have to click on the number you wish to route, and select the end SIP peer to route those calls to, in my case SIP Peer 79908.

At this point you should be able to see the calls come in on the ITSP call logs and the FreeSwitch via sngrep, but it won’t be routed anywhere according to FreeSwitch’s dial plan so…

nano /etc/freeswitch/dialplan/public/1204666xxxx.xml

Now I don’t think you have to name it this way, pretty sure you can name it differently but this is for simplicity for now. and fill it with:

This should be all that’s required, just do another reloadxml, and dial the number.

Outbound Calls

Now with the current Dial-Plan that’s defaulted 01_example.com.xml it’s using the gateway variable we defined in vars.xml so our only outbound proxy at this point. Since I was able to see the calls hitting the les.net portal but getting denied I decided to give les.net a call to see if maybe they had an idea why.

When checking my SIP peer trunk on the portal which was my FreeSwitch it was registering every 20-30 minutes, it was suggested to drop it to between 60-90 seconds.

So in the gateway settings:

nano /etc/freeswitch/sip-profiles/external/lesnet.xml

thx

sofia profile external restart reloadxml

Turns out that wasn’t the case, I had a hunch the problem was the fact the source was 000000000 as your can see:

so I quickly googled this to see if I could find something.. I found this

“dial and bgdial

If the caller id values are not set, the variables in conference.conf.xml will be used. Specifically, the value for caller-id-number will be used for the number and the value for caller-id-name will be used for the name.

If the conference will be dynamically created as a result of this api call (ie this will be the first participant in the conference) – and the caller id name and number is not provided in the api call – the number and name will be “00000000” and “FreeSWITCH”. This appears to be unaffected by the variables in conference.conf.xml.”

Ohhhhhhhh… ok so if I set the call outbound number in the user file…

This works for the single user, to make it more of a NAT like you do with a single public IP address and want to share the internet, you set this variable in the vars.xml file.

and sure enough :D…

Yes!! hahah finally.

That’s it for now! Next round I’ll cover IVRs and all the other fun stuff. This is just the basics. and even then doesn’t cover it very well, just enough to get it all to work. I also noticed that I didn’t have to the NAT rules or the security rules so just the basic NAT is required for FreeNAS and the phones I guess… hahaha

 

Even More PowerShell Fun

The Story

It’s another day, and we all know what that means… yes, another blog post, and even more PowerShell! Can you feel all the power!?!?!

This time it came down to the storage size of my Exchange servers C:\ which turns out to be due to Logs. Logs are great, and best practice is to only clear them if you have a backup copy. Often is the case that logs can be truncated after a backup via VSS by many backup solutions however in my case I could and probably should get that validated with Veeam (as I can’t seem to get that working ‘out of the box’) at the moment. So instead I wanted to know what was “usually” done server side even if someone was not implementing a backup solution.

Source: https://social.technet.microsoft.com/wiki/contents/articles/31117.exchange-201320162019-logging-clear-out-the-log-files.aspx

Neat, but the script is just alright, good for them doing what they want and that’s running it as a scheduled task. Not my goal, but a great source and starting point… let’s have some fun and give this script some roids, much like my last one… I’ll give this a home on GitHub.

Things learned…

  1. Working with the Registry
  2. Determining if Elevated (This is great and I may have a solution to the conundrum in my previous PowerShell post)
  3. Getting a Number, and validating it
  4. Validating Objects by Type
  5. Getting Folder Sizes

Check out my script for all the fun coding bits. I’m a bit tired now as it’s getting late so not much blogging, all more coding. 🙂

ErrorAction Stop Not Stopping Script

Quick Educational note (Source)

$ETLLogKey2 = 'HKLM:\SOFTWARE\Microsoft\Search Foundation for Exchange\Diagnostics'
try{Get-ItemProperty -Path $ETLLogKey2 -ErrorAction Stop}
catch{Write-Host "No Key"}
Write-Host "This should not hit"

Produces:

Well poop… The catch block was triggered but the script did not stop…

Oddly, changing to Throw, which is ugly does make the script stop…

$ETLLogKey2 = 'HKLM:\SOFTWARE\Microsoft\Search Foundation for Exchange\Diagnostics'
try{Get-ItemProperty -Path $ETLLogKey2 -ErrorAction Stop}
catch{throw "No Key"}
Write-Host "This should not hit"

Nice it worked this time, but it’s ugly…

Write-Error is just as ugly, but doesn’t stop the script?

$ETLLogKey2 = 'HKLM:\SOFTWARE\Microsoft\Search Foundation for Exchange\Diagnostics'
try{Get-ItemProperty -Path $ETLLogKey2 -ErrorAction Stop}
catch{Write-Error "No Key"}
Write-Host "This should not hit"

Produces:

Yet if I follow Sages answer in the source, and do a script variable for the stop action it then works???!?!

$ErrorActionPreference = [System.Management.Automation.ActionPreference]::Stop
$ETLLogKey2 = 'HKLM:\SOFTWARE\Microsoft\Search Foundation for Exchange\Diagnostics'
try{Get-ItemProperty -Path $ETLLogKey2}
catch{Write-Error "No Key"}
Write-Host "This should not hit"

Those are really weird results, but all still ugly… so it seems even though -ErrorAction Stop causes a non-terminating error to be treated as a terminating error, but depending on what you do in the catch block determines if there’s a break/exit event being done. In my case to have things look nice and actually stop the script I have to do the follow.

$ETLLogKey2 = 'HKLM:\SOFTWARE\Microsoft\Search Foundation for Exchange\Diagnostics'
try{Get-ItemProperty -Path $ETLLogKey2 -ErrorAction Stop}
catch{Write-host "No Key";break}
Write-Host "This should not hit"

Which finally produced the output I wanted. (I could have also used exit in place of break)

Finally!

Validating URLs:

Source:

Good but broken due to the match on the scheme extension:

function isURI($address) {
	($address -as [System.URI]).AbsoluteURI -ne $null
}

function isURIWeb($address) {
	$uri = $address -as [System.URI]
	$uri.AbsoluteURI -ne $null -and $uri.Scheme -match '[http|https]'
}


isURI('http://www.powershell.com')
isURI('test')
isURI($null)
isURI('zzz://zumsel.zum')

"-" * 50

isURIWeb('http://www.powershell.com')
isURIWeb('test')
isURIWeb($null)
isURIWeb('zzz://zumsel.zum')
isURIWeb('hp:') #Return True

 

Better results with:

function isURI($address) {
($address -as [System.URI]).AbsoluteURI -ne $null
}

function isURIWeb($address) {
$uri = $address -as [System.URI]
$uri.AbsoluteURI -ne $null -and $uri.Scheme -match "http|https"
}


isURI('http://www.powershell.com')
isURI('test')
isURI($null)
isURI('zzz://zumsel.zum')

"-" * 50

isURIWeb('http://www.powershell.com')
isURIWeb('test')
isURIWeb($null)
isURIWeb('zzz://zumsel.zum')
isURIWeb('hp:') #Return True

Outlook and the Cache Mode

The Story

Wouldn’t be another post without another day of annoyances… yup, just another day. So recently it’s been reported that when users are working remotely their outlook decides not to open….

Now since they are remote, I have noticed this only happens if the Outlook client is configured in “Online mode”, instead of “Exchange Cached Mode”… see this MS Docs for more information on the different types and when to utilize each mode.

Originally I configured Online Mode, as most users are locally available in the work network and this is not a major problem. Also when using “Cache mode” not all emails show up in Outlook right away, specially if using folders, there generally shows a link “Click here to view more on Microsoft Exchange” this means items that were not cached (depending on the cache time slider this may vary, I choose 12 months), view this support if you can’t see the link in cache mode enabled.

I used to have an issue with this setting, user reported items wouldn’t load even after clicking the link, however I haven’t seen this to be an issue anymore, so I recommend to enable Outlook Cache mode unless you fall under the other points in MS’s Doc linked above.

Finding Users That Are Not Using Cache Mode

So I was now on a new mission… “How do I know who’s not using Cache mode on outlook?” And this is were the rabbit hole began….

First result, same question

Craig Harts reply…

his or response seem like it’s rather easy until…

wtf is this…? Depreciated?!?! C’mon, but there’s an alternative now that’s better right? No…. Thanks MS… seriously… thanks…

So this older post goes over alternatives, not because they didn’t have access to the cmdlet above, but simply cause they didn’t trust the results.. huh…

Problem is this seem to be reg keys used by older outlook, and new outlook seems to use alternative keys… This sure is fun! However, thanks to others that blog and code in their spare time as well, in this case thanks to Jose Espitia much like in my last blog post this is a great start, but usual me, I don’t like expecting anyone to change the source code. In this case you have to provide a file with a computer list, output path, but it’s hard coded…. OK you know what that means! Yeah I usually would create a new GitHub Repo but first…

Much like the older post mentioned they choose to target end user directly to get the most “accurate data”. This however assumes four things:

  1. That Exchange admins are workstation admins, and have elevated rights on all machines.
  2. That all firewalls are configured and permissions to allow remote querying.
  3. That the user’s are in fact online at the time the query is made.
  4. That RemoteRegistry Service is started and running on end machines.

In my Case it was not and disabled on all machines, I didn’t feel it was beneficial enough to introduce a risk for the simple sake of determining a users connection mode on Outlook.

So I decided to go back to Option A from the old original post, parsing all the RPC Access logs… Using this an alternative reference.

Well I gave it a try and looking through all th elogs there was no reference to “Classic” so I guess that’s no longer valid option either.

Looks like I’m stuck on this one, I can’t seem to find a valid way to find this information out server side with MS removal of the Get-LoggingStatistics cmdlet. And attempting to query all ends users has to many restrictions/hoops that I do not wish to implement. In this case I have to simply go around to all users machines, or wait for the to complain when they work remotely.

Thanks Microsoft I really love what your doing for SysAdmins. Taking away mark and putting him on as COO of Azure so all our beautiful tools from SysInternals are now just the way they are, and new tools, don’t need em right, just buy your cloud subscriptions and who needs SysAdmins… 😛

Anyway… that’s it for today. Sorry no advanced scripts form this post, just use Jose’s script if you wish to query end users machines. Just ensure you have RemoteRegistry service running on all end users machines, and not blocking it in the firewall as well. Hoops I have no interest in jumping through.

Cheers!

WMIC Fun!

I’ve blogged about WMI before, more for setting up dedicated accounts for monitoring purposes.

Today we are going to have some fun with WMIC, the command line interface for simple and quick query data.

I got these ideas after reading this source blog… and I was curious at what level these worked (admin or not)

Using WMI

Most WMIC commands are issued in the following format:

wmic [Object Class] [Action] [Parameters]

For example, you can collect a list of groups or users on the local system and domain using the following commands:

wmic group list brief
wmic useraccount get name,sid

Yup, SIDs are no secret and you can pretty much query the whole domain if there’s been no hardening done. I haven’t tested this on a hardened domain but out of the box all users login name and SID are open for any standard user to query.

You can also perform the same data collection over the network without ever logging into the remote machine provided you know have some administrative credentials that the remote system will accept.

The same command issued against a remote system in another domain looks like this:

wmic /user:"FOREIGN_DOMAIN\Admin" /password:"Password" /node:192.168.33.25 group list brief

I can’t test this in my lab as I don’t have an alternative domain to play with (yet), but let’s see if I can query a member server using a standard domain account:

wmic /node:subca.zewwy.ca group list brief

nope well that’s good…

Processes
WMIC can collect a list of the currently running processes similar to what you’d see in “Task Manager” using the following command:

wmic process list
wmic process get name

Note that some of the WMIC built-ins can also be used in “brief” mode to display a less verbose output. The process built-in is one of these, so you could collect more refined output using the command:

wmic process list brief

Yup, those all work, even as standard user.

Some examples

Start an Application

wmic process call create "calc.exe"

Yeah… that worked…

I decided to see if I could somehow exploit these to get elevated rights, so far no dice.. but I did find this randomly while searching for a possible way…

sure enough, if you add start cmd.exe /k “net use” and name it net use.bat it will go into and endless loop. Mhmm interesting and easiest way to do a Denial Of Service attack.

anyway moving on…

System Information and Settings

You can collect a listing of the environment variables (including the PATH) with this command: (standard User works)

wmic environment list

OS/System Report HTML Formatted

wmic /output:c:os.html os get /format:hform

This was literally cause my standard account didn’t have access to C:\temp cause I created the folder using my admin account at some earlier point in time.

Products/Programs Installed Report HTML Formatted

wmic /output:c:product.html product get /format:hform

Turn on Remoted Desktop Remotely

Wmic /node:"servername" /user:"user@domain" /password: "password" RDToggle where ServerName="server name" call SetAllowTSConnections 1

Get Server Drive Space Usage Remotely (any node commands require elevated permissions, standard user fails at these generally)

WMIC /Node:%%A LogicalDisk Where DriveType="3" Get DeviceID,FileSystem,FreeSpace,Size /Format:csv MORE /E +2 >> SRVSPACE.CSV

Get PC Serial Number (works as standard user)

wmic bios get serialnumber

Get PC Product Number (works as standard user)

wmic baseboard get product

Find stuff that starts on boot (works as standard user)

wmic STARTUP GET Caption, Command, User

Reboot or Shutdown (works as standard user)

wmic os get buildnumber
wmic os where buildnumber="2600" call reboot

Get Startup List (works as standard user)

wmic startup list full

Information About Harddrives (works as standard user)

wmic logicaldisk where drivetype=3 get name, freespace, systemname, filesystem, size, volumeserialnumber

Information about OS (works as standard user)

wmic os get bootdevice, buildnumber, caption, freespaceinpagingfiles, installdate, name, systemdrive, windowsdirectory /format:htable > c:osinfo.htm

User and Groups

Local user and group information can be obtained using these commands:

wmic useraccount list
wmic group list
wmic sysaccount list

For domain controllers, this should provide a listing of all user accounts and groups in the domain. The “sysaccount” version provides you with system accounts built-in and otherwise,which is useful for any extra accounts that may have been added by rootkits.

Identify any local system accounts that are enabled (guest, etc.)

wmic USERACCOUNT WHERE "Disabled=0 AND LocalAccount=1" GET Name

Number of Logons Per USERID

wmic netlogin where (name like "%skodo") get numberoflogons

Get Domain Names And When Account PWD set to Expire

WMIC UserAccount GET name,PasswordExpires /Value

Patch Management

Need to know if there are any missing patches on the system? WMIC can help you find out with this command:

wmic qfe list

The QFE here stands for “Quick Fix Engineering”.
The results also include the dates of install should that be needed from an auditing standpoint.

Shares

Enumeration of all of the local shares can be collected using the command:

wmic share list

The result will also include hidden shares (named with a $ at the end).

Find user-created shares (usually not hidden)

wmic SHARE WHERE "NOT Name LIKE '%$'" GET Name, Path

so far all these are working as standard user, but that doesn’t mean anything.

Networking

Use the following command to extract a list of network adapters and IP address information:

wmic nicconfig list

Get Mac Address:

wmic nic get macaddress

Update static IP address:

wmic nicconfig get description, index
wmic nicconfig where index=9 call enablestatic("192.168.16.4"), ("255.255.255.0")

Yup got to be an admin for that one

Change network gateway:

wmic nicconfig where index=9 call setgateways("192.168.16.4", "192.168.16.5"),(1,2)

Enable DHCP:

wmic nicconfig where index=9 call enabledhcp

Get List of IP Interfaces

wmic nicconfig where IPEnabled='true'

Services

WMIC can list all of the installed services and their configurations using this command:

wmic service list

The output will include the full command used for starting the service and its verbose description.

Other examples

Service Management

 wmic service where caption="DHCP Client" call changestartmode "Disabled"

Look at services that are set to start automatically

wmic SERVICE WHERE StartMode="Auto" GET Name, State

Services Report on a Remote Machine HTML Formatted:

wmic /output:c:services.htm /node:server1 service list full / format:htable

Get Startmode of Services

Wmic service get caption, name, startmode, state

Change Start Mode of Service:

wmic service where (name like "Fax" OR name like "Alerter") CALL ChangeStartMode Disabled

Get Running Services Information

Wmic service where (state="running") get caption, name, startmode, state

Another interesting feature of WMIC is its ability to record the run-time command executed and runtime configuration all in one XML file. A recorded session might look something like this:

wmic /record:users_list.xml useraccount list

Of course, since WMIC wasn’t designed as a recording device, there are some caveats to using the XML. First, you can only use XML output, there are no other formats defined.

Event logs

Obtain a Certain Kind of Event from Eventlog

wmic ntevent where (message like "%logon%") list brief

Clear the Eventlog

wmic nteventlog where (description like "%secevent%") call cleareventlog

Retrieve list of warning and error events not from system or security logs

WMIC NTEVENT WHERE “EventType < 3 AND LogFile != ‘System’ AND LogFile != ‘Security’” GET LogFile, SourceName, EventType, Message, TimeGenerated /FORMAT:”htable.xsl”:” datatype = number”:” sortby = EventType” > c:appevent.htm

Thanks Andrea!

Requesting, Signing, and Applying internal PKI certificates on VCSA 6.7

The Story

Everyone loves a good story. Well today it begins with something I wanted to do for a while but haven’t got around to. I remember adjusting the certificates on 5.5 vCenter and it caused a lot of grief. Now it may have been my ignorance it also may have been due to poor documentation and guides, who knows. Now with VMware now going full linux (Photon OS) for the vCenter deployments (much more light weight) it’s still nice to see a green icon in your web browser when you navigate the nice new HTML5 based management interface. Funny that the guide I followed, even after applying their own certificate still had a “not secure” notification in their browser.

This might be because he didn’t install his Root CA certs into the computers trusted CA store on the machine he was navigating the web interface from. However I’m still going to thank RAJESH RADHAKRISHNAN for his post in VMArena. it helped. I will cover some alternatives however.

Not often I do this but I’m lazy and don’t feel like paraphrasing…

VCSA Certificate Overview

Before starting the procedure just a quick intro for managing vSphere Certificates, vSphere Certificates can manage in two different modes

VMCA Default Certificates

VMCA provides all the certificates for vCenter Server and ESXi hosts on the Virtual Infrastructure and it can manage the certificate lifecycle for vCenter Server and ESXi hosts. Using VMCA default the certificates is the simplest method and less overhead.

VMCA Default Certificates with External SSL Certificates (Hybrid Mode)
This method will replace the Platform Services Controller and vCenter Server Appliance SSL certificates, and allow VMCA to manage certificates for solution users and ESXi hosts. Also for high-security conscious deployments, you can replace the ESXi host SSL certificates as well. This method is Simple, VMCA manages the internal certificates and by using the method, you get the benefit of using your corporate-approved SSL certificates and these certificates trusted by your browsers.

Here we are discussing about the Hybrid mode, this the VMware’s recommended deployment model for certificates as it procures a good level of security. In this model only the Machine SSL certificate signed by the CA and replaced on the vCenter server and the solution user and ESXi host certificates are distributed by the VMCA.

I guess before I did the whole thing, were today I’m just going to be changing the cert that handles the web interface, which is all I really care about in this case.

Requirements

  • Working PKI based on Active directory Certificate Server.
  • Certificate Server should have a valid Template for vSphere environment
    Note : He uses a custom template he creates. I simply use the Web Server template built in to ADCS.
  • vCenter Server Appliance with root Access

Requesting the Certificate

Now requesting the certificate requires shell access, I recommend to enable SSH for ease of copying data to and from the VCSA as well as commands.

To do this log into the physical Console of the VCSA, in my case it’s a VM so I opened up the console from the VCSA web interface. Press F2 to login.

Enable both SSH and BASH Shell

OK, now we can SSH into the host to make life easier (I used putty):

Run

 /usr/lib/vmware-vmca/bin/certificate-manager

and select the operation option 1

Specify the following options:

  • Output directory path: path where will be generated the private key and the request
  • Country : your country in two letters
  • Name : The FQDN of your vCSA
  • Organization : an organization name
  • OrgUnit : type the name of your unit
  • State : country name
  • Locality : your city
  • IPAddess : provide the vCSA IP address
  • Email : provide your E-mail address
  • Hostname : the FQDN of your vCSA
  • VMCA Name: the FQDN where is located your VMCA. Usually the vCSA FQDN

Once the private key and the request is generated select Option 2 to exit

Next we have to export the Request and key from the location.

There are several options on how to compete this. Option 1 is how our source did it…

Option 1 (WinSCP)

using WinSCP for this operation .

To perform export we need additional permission on VCSA , type the following command for same

chsh -s /bin/bash root

Once connected to vCSA from winscp tool navigate the path you have mentioned on the request and download the vmca_issued_csr.csr file.

Option 2 (cat)

Simple Cat the CSR file, and use the mouse to highlight the contents. Then paste it into ADCS Request textbox field.

Signing The Request

Now you simply Navigate to your signing certificate authorizes web interface. usually you hope that the PKI admin has secured this with TLS and is not just using http like our source, but instead uses HTTPS://FQDN/certsrv or just HTTPS://hostname/certsrv.

Now we want to request a certificate, an advanced certificated…

Now simply, submit and from the next page select the Base 64 encoded option and Download the Certificate and Certificate Chain.

Note :- You have to export the Chain certificate to .cer extension , by default it will be PKCS#7

Open Chain file by right click or double click navigate the certificate -> right click -> All Tasks -> export and save it as filename.cer

Now that we have our signed certificate and chains lets get to importing them back into the VCSA.

Importing the Certificates

Again there are two options here:

Option 1 (WinSCP)

using WinSCP for this operation .

To perform export we need additional permission on VCSA , type the following command for same

chsh -s /bin/bash root

Once connected to vCSA from winscp tool navigate the path you have mentioned on the request and upload the certnew.cer file. Along with any chain CA certs.

Option 2 (cat)

Simply open the CER file in notepad, and use the mouse to highlight the contents. Then paste it into any file on the VCSA over the putty session.

E.G

vim /tmp/certnew.cer

Press I for insert mode. Right click to paste. ESC to change modes, :wq to save.

Run

 /usr/lib/vmware-vmca/bin/certificate-manager

and select the operation option 1

Enter administrator credentials and enter option number 2

Add the exported certificate and generated key path from previous steps and Press Y to confirm the change

Custom certificate for machine SSL: Path to the chain of certificate (srv.cer here)
Valid custom key for machine SSL: Path to the .key file generated earlier.
Signing certificate of the machine SSL certificate: Path to the certificate of the Root CA (root.cer , generated base64 encoded certificate).

Piss what did I miss…

That doesn’t mean shit to me.. “PC Load letter, wtf does that mean!?”

Googling, the answer was rather clear! Thanks Digicert!

Since I have an intermediate CA, and I was trying either the Intermediate or the offline it would fail.. I needed them both in one file. So opened each .cer and pasted them into one file “signedca.cer”

Now this did take a while, mostly around 70% and 85% but then it did complete!

Checking out the web interface…

Look at that green lock, seeing even IP listed in the SAN.. mhm does that mean…

Awwww yeah!!! even navigating the VCSA by IP and it still secure! Woop!

Conclusion

Changing the certificate in vCenter 6.7 is much more flexable and easier using the hybird approach and I say thumbs up. 😀 Thanks VMware.

Ohhh yea! Make sure you update your inventory hosts in your backup software with the new certificate else you may get error attempting backup and restore operations, as I did with Veeam. It was super easy to fix just validate the host under the inventory area, by going through the wizard for host configuration.

NTFS Permissions and the Oddities

NTFS Permissions

What is NTFS?

NTFS is a high-performance and self-healing file system proprietary to Windows NT, 2000, XP, Vista, Windows 7, Windows 8, Windows 10 desktop systems as well as commonly used on Windows Servers 2016, 2012, 2008, 2003, 2000 & NT Server. NTFS file system supports file-level security, transactions, encryption, compression, auditing and much more. It also supports large volumes and powerful storage solution such as RAID/LDM. The most important features of NTFS are data integrity (transaction journal), the ability to encrypt files and folders to protect your sensitive data as well as the greatest flexibility in data handling.

Cool, now that we got that out of the way, file systems require access controls, believe it or not that’s controlled using lists called Access Control Lists (ACLs). Huh, who would of thunk it, ACLs either Allow or Deny permissions to the files and folders in the file system.

So far nothing odd or crazy here… There can come times when a user may have multiple permissions on a resource from alternative sources E.G. (Explicit vs Inherited), now depending which will determine whether the action is allowed or dined based on precedence.

A little more intricate, but still nothing odd here. However good reference material. Up Next, another tid bit required to understand the oddtites I will discuss.

File Explorer (explorer.exe)

If you’re an in-depth sysadmin you may know that by default (Windows7+) you can not run file explorer (explorer.exe) as an admin, or elevated. References one and two. Now in the second one there is a work around but I have not tested this, though I will actually probably for my next blog post. But for now the main thing to no is that you can’t run explorer elevated by default.

Turns out this is due to Explorer.exe being single threaded.. apparently.

Source One (says it’s possible, with person reply… didn’t work, links to source 2)

Source Two (Follow up initial question as to why it didn’t work, links to source 3)

Source Three (Old MS doc from unknown author with slight misconception based on my findings below.

“When running as a administrator with UAC enabled, your standard user token does not have write permissions to these protected folders.” –Correct

“Unfortunately, because Windows Explorer was not designed to run in multiple security contexts in the same desktop session, Windows cannot simply throw up a UAC prompt and then launch an elevated instance of Explorer.” –Correct

“Instead, you get one or more elevation prompts (if full-prompting is enabled) and Windows completes the operations using the full administrator token. This can be annoying if you have to make repeated operations in these folders.” –Slightly bad wording, it SHOULD simply utilize UAC prompt creds to complete the requested action (create folder, or navigate folder), but as shown below it will actually adjust the ACL’s themselves to let the action requested complete under the security context of the current running user.

Next! See all Examples of my claim as indicated in this blog post.

User Access Control (UAC)

So again talking WIndows 7 onward here Microsoft made NTFS more secure by having the OS utilize User Access Controls, for when elevated rights were required. For we all do best practices and use different admin and standard accounts, right? To keep it short the lil pop up asking “Are you sure you want to run this?” if you have the ability to run elevated or a Credential Pop-up dialog if you do not.

You can view the “Tasks that trigger a UAC prompt” section of the wiki to get an idea when. (Pretty much anytime you require an system level event)

However I’m going to bring attention this specific one:

Viewing or changing another user’s folders and files

Oddity #1

This brings up our first oddity. If I were to ask you the following question:

You are logged on as an admin on a workstation, you open file explorer, you navigate to a folder in which you do not have either explicit or inherited permissions. When you double click this folder you are presented with a UAC prompt, what does clicking “Continue” do?

A) Clicking Continue causes UAC to temperately runs explorer elevated and navigates into the folder.

B) Clicking Continue will take the current logged on user Security Identifier (SID) and append it to the folders ACL.

Now if you are following along closely we already discussed that A) isn’t even a viable option which means the answer is non other then B…

 

Yup, marvel at it… dirty ACLs everywhere. Now do note I had to break inheritance from the parent folder in order to restrict normal access, which makes sense when your navigating folders in file explorer as an admin already. But this information is still good to know if you do come across this when you are working in an elevated user session.

Also note IF the folder’s owner is SYSTEM or TrustedInstaller, clicking continue will not work and you’ll get an error, cause this action will not take ownership of a folder only grant access, and without the rights to grant those permissions it will still fail, even though there’s nothing stopping you from using takeown or the file explorer to actually grant your account ownership.

Oddity #2

This is the one I really wanted to cover in this blog post. You may have noticed that I stated I broke inheritance, this is generally not best practice and should be done as a last resort usually when it comes to permission management. However it does come around as a solution to access control when it really needs to be super granular.

I had created a TechNet post asking how to restore Volume ACLs, to which no good answers came about. So what I ended up doing was simply adding a new disk to a VM and checked out it’s permissions.

Now if you look closely you’ll notice 3 lines specifying specific access rights for the group “Users”. Now on a workstation, these permissions make perfect sense, a user has the right to read and execute files (needed just to use the system), create folders and they are the owners of them (what good is a workstation if you can’t organize your work), create files and write data (what goods a workstation if you can’t save your work).

However you might think, bah this will be a server (I’ll harden it that standard users can’t have interactive log on) so along with traversal bypass granted by default users should have access to only the specific folders in which they are explicitly granted, and by default will not have any access right inherited.

Removing Users still leaves the Administrators group with full Control rights, and you are a member of that group by domain inheritance, so all is good right? Sounds gravy until…. you realize as soon as you removed the “Users” accounts from the ACLs your admin account has inherited access rights revoked?

Inside the disk was a folder “Test” as you can see by its inherited ACLs

Now this is where it gets weird, it would be safe to assume that my domain admin account which I’m logged in as is part of the Built in administrators group… as demonstrated by this drawing here:

Which is also proven by the fact I can run CMD and other applications elevated via the UAC prompt and I simply click Yes instead of getting a credential box.

Now wouldn’t it be safe to assume that since Administrators have Full Control on the folder in question clearly shows that above, we should be able to traverse the folder, right? It’s basic operation of someone with “Full Control”… and…. awwwww would you look at that? Just look at it! Look at it!

It’s a big ol’ UAC prompt, now why would we get that if we have inherited permission… we already know what it’s going to do… that’s grant my account’s SID permissions, but why? I have inherited full control through administrators don’t I? and sure enough, clicking Continue…

well that’s super weird. I’m skip paste a lot of my trial and error tasks and make the claim, it literally comes down to one ACL that magically makes inheritance work like it’s suppose to…

believe it or not that’s it…. that’s the magical ACL on a folder that will make File Explorer actually adhere to inherited permissions. literally… granting S-1-5-32-545 Users “List folder \ Read Data” permission on the folder, and now as an admin I can traverse the folder without a UAC prompt, and without explicit permissions…

Oddity #3

So I’m like, alright, I’m liking this, I’m learning new things, things are getting weird…. and I can like weird, so I decided like YO! let’s create some folders and like see how things play out when I dickery do with those nasty little ACLs you know what I mean?

 

This stuffs too clean, you know what I mean, all nicely inherited, user owner, nah let’s change things up on this one, SYSTEM you got ownership, and you know what… all regular users.. yer gone you know what that means… inheritance who needs that. This is security, deeerrrrr…..

Awww yeah, and sure enough, trying to traverse the folder gives a UAC prompt, and grants my account explicit permissions, there goes those clean ACLs.

Answer to the Whole Thing

Turns out I was thinking about this all day at work, I couldn’t get it. It honest felt like somehow all access rights were being granted by the “Users” group only…. as if… they are.. using the lowest common denominator… like it can’t… run elevated! DOH!

The answer has been staring me in the face the whole freaking time!

I already stated “If you’re an in-depth sysadmin you may know that by default (Windows7+) you can not run file explorer (explorer.exe) as an admin, or elevated.”

I’m expecting to do task via explorer through an account I have inheritance from BUT the group I’m expecting to grant me the right is an elevated rights group “Administrators”… like DOH!

So the easy fix is create any random security group in the domain, add users accordingly into that group and grant that group full control over the folder, sub-folders and files (even make the group the owner of said folders and subfolders). Then sure enough everything works as expected.

For Example

added my admin account into this group. Then on the file server. Leave the D:\ disk permissions in place. Create a Folder in which other folders can be created and shared accordingly, in this case, teehee let’s call it DATA.

Sure enough, no surprise it looks like this…

everything as it should be, I created the Folder, my accounts the owner, I have inherited Full Control because I am the owner, and all other permissions have been granted by the base disk, besides the one permission which was configured at the disk level to be “this folder only” so all is good.

And now I did some quick searching on how to restrict access without breaking inheritance, and overall most responses was “even though it’s best practice to not break inheritance, alternative means for access control via deny’s is even more dirty”.

So, here we go lets break the inheritance from the disk and remove all users access, now as we discovered we will initially get UAC prompts if we try to navigate it with our admin account after this. Let’s not do that just yet after. So it’s now like this (we granted the group above ownership).

Now since I am a member of this group (I just added my account so I’m going to log off and back on to ensure my group mappings update properly for my kerberos tickets (TGT baby) to work.

whoami /groups

I’m so glad I did this, cause my MMC snap-in did not save the changes and I was not in this group after my first re-logon and sure enough after I fixed it.. 🙂

Now if I navigate the folder I should not get a UAC prompt cause my request to traverse the folder will be granted via File Share Admins, which is not an elevated SID request and I’ll be able to create files and folders without interruptions… lets try..

And there it is, no UAC prompt, all creation options available, and no users in the folders ACLs! Future Admins will need to be added to this group however, if an admin (domain admin or otherwise) attempts to login and navigate this folder they will get a UAC prompt and their SIDs will be auto appended to all folders, subfolders and files! Let me show you…

Welcome DeadUserAdmin! He’s been granted domain admin rights only, and decided to check out the file server…

as shown in the diagram the group permissions, and those inherited by simply being a domain admin, such as local admin. Below the permissions of a file before this domain admin attempts to navigate the folders..

Now as we learnt when this admin double clicks the DATA folder explorer can’t run elevated, and can’t grant traverse access via this accounts nested permissions under the administrators account, and when the UAC prompt appears is granting that SID direct access… lets follow:

There it is! and sure enough…

Yup every folder, and every file now has this SID in it, and when the user no longer works at the company…

SIDE ERROR****

deleting the Users Profile (to fix, naviagte in a couple folders, cut a folder, go to user profile root folder and paste to shorten the overall path name)

So anyway after the user leaves the company and his account gets deleted…

Yay, a whole entire folder/file structure with SIDs as Principals cause AD can’t resolve them anymore. They have been deleted. So how does an admin now fix DeadUserAdmins undesired effects?

Navigate to the root DATA folder properties, Security Tab, advanced settings. Remove the SID…

Be careful of the checkbox at the bottom (Replace all child permissions) use this with caution as it can do some damages if other folders down the line have broken inheritance and specific permissions. In this case all folders and files inherent from this base DATA drive and thus….

All get removed. If there are other folders with broken inheritance then an Audit is required of all folders, their resources, their purposes, and who’s suppose to have access.

Another option is to nest domain admins into file share admins, then it all works well too.

I hope this blog post has helped someone.

Email Scamming

The Story

Everyone loves a good story, ehhhhhhhhhhhh.

Anyway sitting around playing a new puzzle game I picked up The Talos Principle. Enjoying it very much, and I my phone goes off, just another email. Looking at the Subject did have me intrigued (while also instantly alerting me that its a scam). Now I plan to cover this blog post in 2 parts. 1 in which I cover the basics of catching “Red Flags” and how to spot these types of emails for the basic user, and 2 more technically in-depth for those that happen to be admins of some kind. Let’s begin.

The Email in Question

Now looking right at this it may not scream out at you, but I’ll point them all out.

First Red Flag

First off, the Subject, the first thing anyone sees when they get an email, and in this case it’s designated to grab attention. “Order of a Premium Account”? What I didn’t order any premium account. So the inclination is to open the email to find out more. Most of the time this is a safe move to make, but I’m sure hackers could make it in at this point if it was an APT (Advanced Persistent Threat) and they really wanted to target you. In this case, not likely. This in itself isn’t a red flag as many legit emails can be of high importance and the sender could use alerting terms to ensure action is taken when time is of the essence. However it still a tactic used by the perpetrator.

Second Red Flag

So what’s the body tell us? In this case it is a clear and definitive “Red Flag”; Vague, and requesting the user to open an attachment for more details. This is the hugest red flag, the body should contain enough information to satisfy the recipient to understand exactly what an attachment would justify being there for.

Third Red Flag

Now mixing the two together we get another “Red Flag” the subject was for a premium account for a “Diamond Shop App.” whatever that is, I suppose many apps have separate account creations and thus this isn’t exactly alarming, however, if it was from the Apple store the email I’m assuming would either follow Apples template (which this doesn’t), considering the attachment is labeled “Apple Invoice.doc”. I also don’t use the Apple Store so for me was an easy red flag.

Fourth Red Flag

Grammar; “Are you sure to cancel this order, please see attachment for more details. thanks you” a question ending in a period with a following “thanks you” with an s and no cap, and the subject was for an account creation…. need I say more?

What now?

OK, so pretty obvious here there some shenanigans goin’ on here. If you’re an end user this is a good time to send the email (as an attachment) to your IT department. It is important to send the email itself as an attachment to retain the email headers (discussed later in this post) for admins to analyze the original sender details.

Technical Stuff

Now we’re going to get technical, so if you are not a technical person you education session is done, else keep reading.

Initial Analyses

Yeah you guessed it; VirusTotal.

Well, nyet….

Nothing… OK, let’s analyze the headers quick with MxToolbox

Here we can see it was sent from the domain “retail-payment.com”, they also masked their list of targets by BCCing them all, shady, and pointing to main to address to noreply@apple.com or device@apple.com which probably are non existent addresses for apple, and making it look more legit while not letting apple actually know. What about this sending domain?

sad another zero day domain registration, I was expecting GoDaddy to be honest, was rather disappointed to see Wix supporting such rubbish.

What’s next? Joe Sandbox!

At this point it’s clear the file and email are brand new attempts and not caught by virus total, so what is it attempting to accomplish. I signed up to JoeSandbox to find out. Then submitted the file, I was impressed with the results!

Results…

I’m not sure why older OS with older Office was clean? but newer showed some results, when I opened the report I was like HA!

Neat looks like it the doc had links to some websites, and yeah.. the sandbox went there! 😀

Would ya look at that! It looks like the apple login page, thankfully the URL doesn’t match apple’s at all and should be another duh red flag.

OK, who registered that domain?

I have no clue who that registrar is, nor do I know how they managed to keep it alive since the 2000’s hosting malicious phishing sites? Sad…

Conclusion

Don’t open up stupid emails, and report them to your admins whenever possible. 😀

Using OpenSSL to convert PKCS12 to PEM

Found from here

openssl pkcs12 -in path.p12 -out newfile.crt.pem -clcerts -nokeys
openssl pkcs12 -in path.p12 -out newfile.key.pem -nocerts -nodes

After that you have:

  • certificate in newfile.crt.pem
  • private key in newfile.key.pem

To put the certificate and key in the same file use the following

openssl pkcs12 -in path.p12 -out newfile.pem

If you need to input the PKCS#12 password directly from the command line (e.g. a script), just add -passin pass:${PASSWORD}:

openssl pkcs12 -in path.p12 -out newfile.crt.pem -clcerts -nokeys -passin 'pass:P@s5w0rD'

Thanks KMX