Author: Aemilianus Kehler

Posted on

SharePoint Site Slow Load

So I finish up another SharePoint 2010 -> 2016 migration and after themeing and everything is put in place, the next day my dev tells me he notices the sites slow. So I run a couple pages and sure enough each page takes 20-25 seconds to load.

Here’s a Snippet of the network console of a site that works normally. Full Site Load in almost under a second.

Not sure what was going on since the test commands were 100% clean before hand and tested this a couple times in a test, followed all my documentation to a tee.

Here’s the snippet of the slow loading SharePoint. *Note I updated this Blog post cause I came across this issue again with a newly created SharePoint Site.

As you can see over 10x slower. Trust me, you feel this…

Checking the logs showed error messages of the SharePoint Managed account failing to access the User Profile Database. (Ohhh bad me, I should be a bit more specific, I found these lines of errors in the ULS Logs (%ProgramFiles%\Common Files\Microsoft Shared\Web Server Extensions\16\Logs)

Checking the permissions set for the managed account in the database (the instance and database name are dependent on your particular SharePoint installation) sure enough showed no access rights, when all other managed services accounts had basic connect rights.

Under security I found the security principal of the service account and under it’s User Mappings I added the DB_Access right for the managed service account. Sure enough this cleared the error message from the SharePoint logs and sites were loading super last. Always check your logs…

Posted on

VMware ESXi boot and the Config

Sadly this post will be really short as again, lots going on. Recovering a host that failed after a regular reboot, which had a superblock corruption on it’s main OS drive. Also, the BELK series will be done, I just need a bit more time. Sorry for the delays.

“Failed to load /sb.v00” [Inconsistent Data]

Since this drive was not on the main datastore on the host all the VMs were unaffected.

Now loading linux showed the drive data was till accessible, but I also had a feeling this USB drive was on it’s way out. I created a copy using DD, *sadly I didn’t do it the smart way and place it on a drive big enough to save it as a image file, but instead directly to another drive of the same size.

I tried to install the same image of ESXi on top of the current one in hopes it would fix the boot partition files along the way. This only made the host get past /sb.v00 and vault randomly past it with “Fatal Error: 6 [Buffer Too Small]”

I was pretty tired at this point since the server boot times are rather long and attempts were becoming tedious. I did another DD operation of the drive, to the same drive (still not having learned my lesson) and when I awoke to my dismay, it failed only transferring 5 gigs with an I/O error. This really made me sure the drive was on the way out, but it was still mountable (the boot partitions 5, 6 and 8)

At this point you might be wondering, why doesn’t he just re-install and reload a backup config? Which is fair question, however one was not on hand, but surely it must be somewhere on the drive. I know how to create and recover on a working host but a one that can’t boot? Then I found this gem.

Now through out my attempts I did discover the boot partitions to be 5 and 6 and I did even copy them from a new install to my copied version I made about and it did boot but was a stock config. I was stumped till I read the section from the above blog post on “How to recover config from a system that doesn’t boot”. Line 7 was what nailed it on the head for me:

“mount /dev/sda5 /mnt/sda5

7. In the /mnt/sda5 directory, you can find the state.tgz file that contains ESXi configuration. This directory (in which state.tgz is stored) is called /bootblank/ when an ESXi host is booted.”

I was just like … wat? That’s it. Grabbed the bad main drive mounted on a linux system, saw the state.tgz file and made a copy of it, connected the new drive that had a base ESXi config, replaced the state.tgz file with the one I copied, booted it and there was the host in full working state with all network configs and registered VMs and everything.

Not sure why the config is stored in the boot partition, but there you go. Huge Shout out to Michael Bose for his write I suggest you check it out. I have saved it case it disappears from the internet and I can re-publish it. For now just visit the link. 🙂

Posted on

BELK Stack on Docker
(Part 2 – Docker Compose)

The Story Continues

Following on from the last post, today we cover docker-compose to allow for easier deployment of docker images and configurations. As from my previous post you may want to indulge in the same reading I did here.

Past those nice formalities, I find myself missing something… I’m not sure what it could be… oh yeah…. dependencies!

Installing Docker-Compose

Can I use apt-get?

Would seem like it… but

IT’s outdated… 😀

Other way is via pip or the intended way…

Working with Docker-Compose

  • docker-compose ps — lists all the services in a network. This is especially helpful when troubleshooting a service as it will give you the container ID and you can then run docker -it exec <ID> bash to enter the container and debug as needed.
  • docker-compose build — generates any needed images from custom Dockerfiles. It will not pull images from the Docker hub, only generate custom images.
  • docker-compose up — brings up the network for the services to run in
  • docker-compose stop — stops the network and saves the state of all the services
  • docker-compose start — restarts the services and brings them back up with the state they had when they were stopped
  • docker-compose down — burns the entire Docker network with fire. The network and all the services contained within are totally destroyed.

How to Docker-Compose?

The last big question is: how to write a docker-compose.yml, and it’s actually very easy and follows a standard formula.

Here is a template of what any docker-compose.yml will look like.

  • Sample Docker Compose Template
version: "2"
  services:
    <name_of_service>:
      build: <path_to_dockerfile>
OR
      image: <name_of_image:version>
      enviroment:
        - "ConfVar:value"
        - "homeDir:/home/dir"
      ports:
        - "[HostPort]:[ContainerPort]"
        - "80:80"
      volumes:
        - /path/container/will/use

Every docker-compose file will start with a minimum of version: "2", if you’re doing a Docker Swarm file it will need version: "3", but for a single docker-compose.yml, you’ll need v2.

See here for more on the use of volumes

I’m gonna keep this post short and use examples of these first two blogs it part 3. Where I setup and configure the first container in the BELK Stack; Elasticsearch.

See you all at part 3! 😀

Posted on

BELK Stack on Docker
(Part 1 – Docker)

The Story

This time our goal to setup a SEIM (Security Event & Information Monitoring) which will gather data via the BELK Stak (Beat, Elasticsearch, Logstash and Kibana). This is going to take (I’m assuming, as I’ve just started) about 4-5 separate blog posts to get this off the ground.

It has taken me a couple weeks of smashing my head into a wall simply due to my own ignorance, so in this blog series I’m going to cover more step-by-step exactly what needs to be done for my particular setup. There are many ways you can configure services these days, which still includes bare metal. If I so chose I could run Docker on a bare metal Ubuntu server, or even a bare metal windows server, but in this case I’m going to install docker on a Ubuntu server which will happen to be itself a VM (Virtual Machine).

Now with that in mind, here’s some basic reading you probably should do before continuing on. Now before we go on let’s be clear on one thing, docker itself doesn’t run on magic, or fluffly rainbow clouds, as I mentioned in the paragraph above it runs on some system, whether that’s again bare metal or some VM of some kind [Think IaaS (Infrastructure as a Service)], in this blog it will be a Ubuntu VM. The specs of this machine should suffice for the application and workloads that are going to be created on it.

Dockerfile Commands

Below, are the commands that will be used 90% of the time when you’re writing Dockerfiles, and what they mean.

FROM — this initializes a new build stage and sets the Base Image for subsequent instructions. As such, a valid Dockerfile must start with a FROM instruction.

RUN — will execute any commands in a new layer on top of the current image and commit the results. The resulting committed image will be used for the next step in the Dockerfile.

ENV — sets the environment variable <key> to the value <value>. This value will be in the environment for all subsequent instructions in the build stage and can be replaced inline in many as well.

EXPOSE — informs Docker that the container listens on the specified network ports at runtime. You can specify whether the port listens on TCP or UDP, and the default is TCP if the protocol is not specified. This makes it possible for the host and the outside world to access the isolated Docker Container

VOLUME — creates a mount point with the specified name and marks it as holding externally mounted volumes from the native host or other containers.

You do not have to use every command. In fact, I am going to demonstrate a Dockerfile using only FROM, MAINTAINER, and RUN.

Images vs. Containers

The terms Docker image and Docker container are sometimes used interchangeably, but they shouldn’t be, they mean two different things.
Docker images are executable packages that include everything needed to run an application — the code, a runtime, libraries, environment variables, and configuration files.
Docker containers are a runtime instance of an image — what the image becomes in memory when executed (that is, an image with state, or a user process).

Examples of Docker containers. Each one comes from a specific Docker image.
In short, Docker images hold the snapshot of the Dockerfile, and the Docker container is a running implementation of a Docker image based on the instructions contained within that image.

This is true, however this image is a bit misleading as it’s missing the versioning which will become apparent a bit later on in this blog post.

Docker Engine Commands

Once the Dockerfile has been written the Docker image can be built and the Docker container can be run. All of this is taken care of by the Docker Engine that I covered briefly earlier.

A user can interact with the Docker Engine through the Docker CLI, which talks to the Docker REST API, which talks to the long-running Docker daemon process (the heart of the Docker Engine). Here’s an illustration below.

The CLI uses the Docker REST API to control or interact with the Docker daemon through scripting or direct CLI commands. Many other Docker applications use the underlying API and CLI as well.

Here are the commands you’ll be running from the command line the vast majority of the time you’re using individual Dockerfiles.

  • docker build — builds an image from a Dockerfile
  • docker images — displays all Docker images on that machine
  • docker run — starts container and runs any commands in that container
  • there’s multiple options that go along with docker run including
  • -p — allows you to specify ports in host and Docker container
  • -it—opens up an interactive terminal after the container starts running
  • -v — bind mount a volume to the container
  • -e — set environmental variables
  • -d — starts the container in daemon mode (it runs in a background process)
  • docker rmi — removes one or more images
  • docker rm — removes one or more containers
  • docker kill — kills one or more running containers
  • docker ps — displays a list of running containers
  • docker tag — tags the image with an alias that can be referenced later (good for versioning)
  • docker login — login to Docker registry

A big thank you to: Paige Niedringhaus for her contributions you can see most of this theory content was a direct copy paste, but not all the content just the basic relevant ones (in case the source material ever goes down).

Now that we got the theory out of the way, let’s get down to the practical fun!

Installing Docker

https://docs.docker.com/install/linux/docker-ce/ubuntu/

Uninstall old versions🔗

Older versions of Docker were called docker, docker.io, or docker-engine. If these are installed, uninstall them:

$ sudo apt-get remove docker docker-engine docker.io containerd runc

It’s OK if apt-get reports that none of these packages are installed.

Installing Dependencies

Don’t got non moving on…

sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

apt-key fingerprint 0EBFCD88

sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

Install Docker Engine – Community

Update the apt package index.

sudo apt-get update

Install the latest version of Docker Engine – Community and containerd, or go to the next step to install a specific version:

sudo apt-get install docker-ce docker-ce-cli containerd.io

Got multiple Docker repositories?

If you have multiple Docker repositories enabled, installing or updating without specifying a version in the apt-get install or apt-get update command always installs the highest possible version, which may not be appropriate for your stability needs.

To install a specific version of Docker Engine – Community, list the available versions in the repo, then select and install:

List the versions available in your repo:

apt-cache madison docker-ce
docker-ce | 5:18.09.1~3-0~ubuntu-xenial | https://download.docker.com/linux/ubuntu xenial/stable amd64 Packages
docker-ce | 5:18.09.0~3-0~ubuntu-xenial | https://download.docker.com/linux/ubuntu xenial/stable amd64 Packages
docker-ce | 18.06.1~ce~3-0~ubuntu | https://download.docker.com/linux/ubuntu xenial/stable amd64 Packages
docker-ce | 18.06.0~ce~3-0~ubuntu | https://download.docker.com/linux/ubuntu xenial/stable amd64 Packages
...
b. Install a specific version using the version string from the second column, for example, 5:18.09.1~3-0~ubuntu-xenial.
sudo apt-get install docker-ce=<VERSION_STRING> docker-ce-cli=<VERSION_STRING> containerd.io

Verify that Docker Engine – Community is installed correctly by running the hello-world image.

sudo docker run hello-world

Woooo, what a lot of fun…. Just note one thing here…

Executing the Docker Command Without Sudo (Optional)

By default, the docker command can only be run the root user or by a user in the docker group, which is automatically created during Docker’s installation process. If you attempt to run the docker command without prefixing it with sudo or without being in the docker group, you’ll get an output like this:

docker: Cannot connect to the Docker daemon. Is the docker daemon running on this host?.
See 'docker run --help'.

If you want to avoid typing sudo whenever you run the docker command, add your username to the docker group:

sudo usermod -aG docker ${USER}

To apply the new group membership, log out of the server and back in, or type the following:

su - ${USER}

You will be prompted to enter your user’s password to continue.

Confirm that your user is now added to the docker group by typing:

id -nG

If you need to add a user to the docker group that you’re not logged in as, declare that username explicitly using:

sudo usermod -aG docker username

The rest of this article assumes you are running the docker command as a user in the docker group. If you choose not to, please prepend the commands with sudo.

Let’s explore the docker command next. Thanks Brian

Creating your Dockerfile

The first thing we’re going to do is create a new directory to work within; so open a terminal window and issue the command as root…

 mkdir /dockerfiles
chown dadocker:docker /dockerfiles

Change into that newly created directory with the command

 cd /dockerfiles

Now we create our Dockerfile with the command nano Dockerfile and add the following contents:

FROM ubuntu:latest
MAINTAINER NAME EMAIL

RUN apt-get  update && apt-get -y upgrade && apt-get install -y nginx

Where NAME is the name to be used as the maintainer and EMAIL is the maintainer’s email address.

Save and close that file. (In my case i called it dockerfile; with a lowercase d)

Building the Image

Now we build an image from our Dockerfile. This is run with the command (by a user in the docker group):

docker build -t "NAME:Dockerfile" .

Where NAME is the name of the image to be built.

in this case . simply represents the local directory, else specify the path of the file…

Listing Images

docker images

Deleting Images

docker rmi image:tag

Running Images (Creating Containers)

docker run image

well poop, after running and stopping a container I was unable to delete the images… Internets to the rescue! since a force seemed rather harsh way to do it.

By default docker ps will only show running containers. You can show the stopped ones using docker ps --all.

You can then remove the container first with docker rm <CONTAINER_ID>

If you want to remove all of the containers, stopped or not, you can achieve this from a bash prompt with

$ docker rm $(docker ps --all -q)

The -q switch returns only the IDs

yay it worked!

Summary

Most of the time you won’t be directly installing docker, or building your own images, but if you do you at least now know the basics. These will become import in the future blog posts. I hope this helps with the basic understanding.

In the next blog post I’ll cover Docker-Compose, which will allow use to spin up multiple images into a single working container which will be the bases of our ELK stack. 🙂

Posted on

FreeSWITCH – IVRs and Fun Stuff

In my previous post I covered how to setup FreeSWITCH behind a PAN firewall, connect with a gateway (the ITSP) and configure a very basic default and public dialplans to get simple inbound and bound calls working.

For the best reading I suggest this book. 🙂

FreeSwitch

IVRs

The path when working with IVRs is:

/etc/freeswitch/ivr_menus/

inside we can see the following:

root@FreeSwitch:~# ls /etc/freeswitch/ivr_menus/
demo_ivr.xml  my_ivr.xml  new_demo_ivr.xml

not a whole lot. First thing I wanted to figure out was how calling ext 5000 would lead to this IVR.

So looking at the default internal dial-plan, we can see it

nano /etc/freeswitch/dialplan/default.xml
    <!-- a sample IVR  -->
    <extension name="ivr_demo">
      <condition field="destination_number" expression="^5000$">
        <action application="answer"/>
        <action application="sleep" data="2000"/>
        <action application="ivr" data="demo_ivr"/>
      </condition>
    </extension>

Looks like it’s the application IVR data=demo_ivr. Quick way to find out let’s rename the file and see if the dialplan still works..ok, it’s not the filename, but the name value in the xml files located within the folder specified above.

Now as you can see I managed to get the screaming monkeys to work by simply recording a stream of screaming monkeys and exporting it with audacity in compressed ULaw wav format, uploaded it to freeSWITCH via WinSCP. then changed the action type and param value.

Now if you get into the nitty gritty, you’ll notice the default IVR uses phrases which are pieced together pieces of smaller recordings. These you may notice by default are also relatively referenced, instead of fully “ivr/ivr-that-was-invalid-entry.wav” which you may notice from searching exists only in the language folders of the sounds…

for the time being I won’t get into making that custom of an IVR, instead start off simple. I’m gonna create an audio recording of my options (1 for sales, 2 for work, 3 for support, 4 for other). Then used WinSCP to copy to the Freeswitch server, then copied to location.

root@FreeSwitch:/etc/freeswitch/ivr_menus# cp /home/zewwy/ZewwyCA.wav /usr/share/freeswitch/sounds/
root@FreeSwitch:/etc/freeswitch/ivr_menus# ls /usr/share/freeswitch/sounds/
en  es  fr  music  pt  ru  ScreamingMonkey.wav  ZewwyCA.wav

ok now, let’s make the IVR do stuff with these options….

first we’ll set the caller ID name and number in vars.xml so we can use the default variables in our directories, and hopefully for outbound calls.

So with my recording in place, i created my IVR as follows:

nano /etc/freeswitch/ivr_menus/zewwy_ivr.xml

then created a new extension to reach in my default dialplan

nano /etc/freeswitch/dialplan/default.xml

Now I just need to change my public dialplan to call this extension instead of my SPAPhone directly.

Change 1002 to 5006. That’s it. 😀 now got a IVR.

Time of Day Call Routing

I used FreeSwitch’s own post on this as a reference.

For me, I created option one in the options for “Sales”. I don’t want to be bothered about items for sales when I’m sleeping, or off work. Currently as above you can see calls are going straight to my voice mail. Well let’s change that…

<!-- My Cell -->
<extension name="mycell">
  <condition field="destination_number" expression="^mycell$">
    <action application="bridge" data="sofia/gateway/${default_gateway}/1#######"/>
  </condition>
</extension>

First, let’s create a new extension for Sales; 2222: in default dialplan

<extension name="Sales-x2222">
  <condition field="destination_number" expression="^2222$">
    <action application="transfer" data="Sales"/>
  </condition>
</extension>

just below that:

<extension name="Sales" continue="true">
<condition field="destination_number" expression="^Sales$"/>
  <condition wday="2-6" hour="9-18">
    <action application="set" data="Sales_open=true"/>
    <action application="transfer" data="xfer-to-sales"/>    
    <anti-action application="set" data="Sales_open=false"/>
    <anti-action application="transfer" data="xfer-to-sales"/>
  </condition>
</extension>
<extension name="xfer-to-sales">
  <condition field="destination_number" expression="^xfer-to-sales$"/>
  <condition field="${Sales_open}" expression="^true$">
    <action application="transfer" data="mycell"/>
    <action application="answer"/>
    <action application="sleep" data="2000"/>
    <action application="voicemail" data="default ${domain_name} 1002"/>
    <anti-action application="voicemail" data="default ${domain_name} 1002"/>
  </condition>
</extension>

If you can’t tell what is happening here, we are creating a dial number named Sales-x2222 when you dial 2222. Then we define the sales work hours Weekdays from 9 till 6, which defines the normal action lines, in this case it sets a variable “Sales_open” to true, otherwise if not within this time, set the variable to false. In the third area we use this flag to either call mycell or leave a voicemail on ext 1002.

Now I simply changed the first line in my IVR instead of bridging a call to my cell, I send it to ext 2222 which will only call my cell during working hours. 🙂

*NOTE* I could have also simply done this directly under the Public based dialplan such as this:

<include>
	<expression name="public_did">
		<condition field="destination_number" expression="1#######$">
			<action application="answer"/>
			<condition wday="2-6" hour="9-18">
				<action application="transfer" data="callfwd">
				<anti-action application="ivr" data="zewwy_ivr">
			</condition>
		</condition>
	</expression>
	<expression name="callfwd">
		<condition field="destination_number" expression="^callfwd$">
			<action application="answer"/>
			<action application="speak" data="flite|rms|Calling someone in reguards to a item for sale. Hold please."/>
			<action application="set" data="effective_caller_id_name=SALE(${caller_id_name})"/>
			<action application="set" data="effective_caller_id_number=${caller_id_number}"/>
			<action application="bridge" data="sofia/gateway/${default_gateway}/1#######"/>
		</condition>
	</expression>
</include>

However I wanted my IVR to stay the same no matter when it was called. So I placed my time of day routing at the internal dial-plan on the specific use case/department. It was simply a more scalable example to use.

Blast Group

here’s an example of a 2 phone blast group via ext 511 using the default dial plan, blasting ext’s 1002 and 1003

this took me a bit…

Hunt Group

ext 512 dials Ext 1001, if not available call 1002 (ring only for 20 second), if not available ring ext 1003 (for 30 seconds), if no answer drop to 1002’s voice mailbox.

some Music on Hold would be nice while the call is being transferred…

    <extension name="a_blast_group">
      <condition field="destination_number" expression="^511$">
        <action application="set" data="ringback=${us-ring}"/>
        <action application="set" data="transfer_ringback=$${hold_music}"/>
        <action application="answer"/>
        <action application="sleep" data="2000"/>
        <action application="bridge" data="sofia/$${domain}/1002,sofia/$${domain}/1003"/>
      </condition>
    </extension>

    <extension name="a_hunt_group">
      <condition field="destination_number" expression="^512$">
        <action application="set" data="ringback=${us-ring}"/>
        <action application="set" data="transfer_ringback=$${hold_music}"/>
        <action application="answer"/>
        <action application="sleep" data="2000"/>
        <action application="set" data="continue_on_fail=true"/>
        <action application="set" data="call_timeout=20"/>
        <action application="bridge" data="sofia/$${domain}/1001|sofia/$${domain}/1002"/>
        <action application="set" data="call_timeout=30"/>
        <action application="bridge" data="sofia/$${domain}/1003"/>
        <action application="voicemail" data="default ${domain_name} 1002"/>
      </condition>
    </extension>

Hope someone found this guide useful. 🙂

Posted on

FreeSWITCH

The Story

My buddy Troy did a presentation, I wanna try it out. So this is going to be a shit show… let’s go…

Sources: Specs

Minimum/Recommended System Requirements
32-bit OS (64-bit recommended)
512MB RAM (1GB recommended)
50MB of Disk Space
System requirements depend on your deployment needs. We recommend you plan for 50% duty cycle.

Install Source for Debian 10

Buddy Troys Presentation

Install Debian 10

So I’ll setup a VM with those nice minimum requirements, could def use the memory savings, most servers these days are redic.

LimbooooooooOOOOOOOOOooo! How low can you go?!

Alright let’s install Debian 10.

Install Source Info and Install Source Image I’ll use the netinstall image.

Mount image to VM… booot er up (I’m gonna try EFI instead of BIOS)

Nice, it booted, Install Graphical or Install, just install, we want to keep it CLI only as it has bare resource allocations.

set root password, create alternative user, guided use entire disk, or set however you like, or however you deploy your AC3 AWS nodes or whatever cloud based instance you might be using. Whatever floats your digital boats.

*Digitized Voice* All your base are belong to us…

ohhh boy…. anyway.

SSH and Standard system packages… this installer keeps going…

Wooo never thought I’d see the day… OK so now that we finally have a clean Debian server, we can move to the next step. 😀

FreeSwitch Install

From Source: “Debian 10 “Buster” is the reference platform for FreeSWITCH™ as of version 1.10

Dependencies are available from FreeSWITCH repository via the “apt-get build-dep freeswitch” command.”

ok let’s try that?

Not sure why that’s at the top of the document when it doesn’t work out of the box, let’s follow along with the “easy way” then…

apt-get update && apt-get install -y gnupg2 wget lsb-release

Moving on…

wget -O - https://files.freeswitch.org/repo/deb/debian-release/fsstretch-archive-keyring.asc | apt-key add -

# you may want to populate /etc/freeswitch at this point.
# if /etc/freeswitch does not exist, the standard vanilla configuration is deployed
apt-get update && apt-get install -y freeswitch-meta-all

Uhhh ok, I don’t have a config in mind per say so I guess I’ll use the predefined one without creating that directory or file… let’s go!

Off she goes 200+ already! That didn’t take too long. Let’s see if we can get into the freeswitch cli…

fs_cli -rRS

[ERROR] fs_cli.c:1565 main() Error Connecting [Socket Connection Error]

OK dokie then, let’s give er a good old reboot. After reboot, haza!

The Presentation in a Nut Shell

SIP (Session Initiation Protocol) -> Initiates the connection for the task
SDP (Session Description Protocol) -> Connection for what
RT(C)P (Real Time (Control) Protocol) -> RTP: Audio Packets RTCP: Metadata

Now slide 25 while very simple topology layout isn’t crazy it was the mentioning of alternative NAT tricks which kind of boggled my mind a bit. The other day I had issues with my Signal app using mobile data excessively even though I was on WiFi. Took me a little while to figure out but it was my firewall that was blocking the traffic and it appears Signal secretly uses any alternative networks on the device to establish the required connection. During the research for a solution, I found a PaloAltoNetworks thread on the issue

Creating a rule with the three main applications (Signal, SSL, STUN) allowing any service ports, and then turning off my mobile data. Still resulted in failed Signal calls. I have to open the rule up completely and even disable server response inspection. I had talked to my local PAN technical rep, I might just make a separate blog post about that entirely. Anyway just making note of that as a possible infrastructure to hurdle while I go through this endeavor…

Check out this Wiki Page on more details on STUN if you have the basic understanding of the difference between TCP and UDP the contents should be fairly easy to digest. However, I digress and move on.

Well it’s going to be harder than I thought to put all this info into a Nut Shell, so instead I’ll try to cover each piece of the puzzle one at a time. First thing on any server is to have a static IP (at least if your behind a NAT which is mentioned many times in his presentation, and I’ll discuss my setup and how that flies when we get to that step). For now let’s just set our internal static private IP address.

OK strange, coming back to this VM from yesterday I was still int the freeswitch CLI, yet typing /exit would bring up the same freeswitch CLI… so hard reboot… and… ok so the initial Debian install guide said to do fs_cli with some options. Read here for a PDF of details options truns out the -R is reconnect when disconnect, and /exit, /bye, /quit are all disconnects. So just use fs_cli without -R, and the /exit works without issue.

Set Host Static IP

Now with that annoyance out of the way, well use this Debian guide to set out IP as root.

nano /etc/network/interfaces

from:

auto eth0
allow-hotplug eth0
iface eth0 inet dhcp

to:

auto eth0
iface eth0 inet static
        address 192.0.2.22/24
        gateway 192.0.2.1

This leaves us so far with this very, very basic network diagram:

Super simple, but doesn’t cover the SIP connections coming in, and the following STUN and NAT traversal attempts, while most home routers may allow these connections the PA I’m behind, not so much. So we’ll again cover those details when we get to them.

*NOTE* I had to make this a Static IP NAT else I wouldn’t get Audio when doing direct sip calls not out to the regular PSTN. See this Post for more details. I also created the custom Apps and app override rules

Managing Users

Slide 33 starts off with:

sudo chmod g+w -R /etc/freeswitch

I’m starting to realize the slide out of context (being there for the presentation) is rather hard to follow along with, but since I’m doing everything as root for now (Since I didn’t add my standard user to the FreeSwitch group) I’ll just ignore this line for now.

  • cd /etc/freeswitch/directory/default
  • Remove everything but 1000.xml, 1001.xml
  • Edit for our user accounts
  • We can reload this configuration without restarting freeswitch

Ok dokie….

Neat! I had no idea you could do that! so do…

rm -v !("1000.xml"|"1001.xml")

That was so easy! 🙂 Since I’m again reading this out of context I’m not sure what edits were made to what files for the line “edit for our user accounts”. Looks like just the XML files we left behind, these appear to be template, as usual XML based, so knowing which fields to edit can be a bit trick to point out.

OK… a bit more details from the main source…

That makes sense… so my final config:

Connecting a Phone

Before we can make any calls we are going to need a phone, now this can be almost any device, a laptop with softphone software , or can be a physical phone as long as it supports the SIP protocol. Lucky for me I have 2 different devices at my disposal for testing. Two older Cisco phones: 1) a Cisco SPA525g and 2) a Cisco WiFi 7925.

The desk phone I feel still looks nice and modern despite it’s age, the 7925 looks like an old brick Nokia and you can imagine the software is just as bad. So let’s see how we can get this into the mix. Lucky for me both support SIP.

Accordingly to the slides on slide 32 looks like we have to define the server listening address and port (which we will leave at the default 5060) that will be the unencrypted default port.

nano /etc/freeswitch/vars.xml

Oh yeah… that default password thing mentioned in slide 31, not sure why this would be clear text in a clear text xml file on a open config path, but… *Smiles n Nods* changing cleartext password.

Since I’m not making any changes at this time, we will just exit and cover applying config changes once we get there. I need to double check some sources to get my head around all this stuff right now, so please bare with me on this blog post, I’m literally learning everything as I go.

7925 – Settings locked (Press **# to unlock settings)
Setup all the WiFi settings then connected to my local network. Use this Cisco Doc for help configuring a Cisco 7925

This is were things kind of went sour. the Cisco 7925g is SCCP or Cisco protocol only (lame) so no SIP. literally Nothing. There maybe a way to use Skippy mod for FreeSwitch but well see about that in a future post.

As for the SPA525g It took me a good while of digging before I figured it out.

  1. Step IP
  2. Log into Web interface http://IP
  3. Click on “Admin Login”
  4. You should now see a SIP tab, leave it, click on Ext 1
  5. Fill in the Proxy Address: (IP of freeswitch)
    User ID: 1001
    Password: (AS set in XML)

I finally managed to get a successful registration after this (but all the soft buttons were lit up and displaying 1001)

So the phone doesn’t look nice (yet) and we only have one, with our 7925 out of the picture for now, I guess I may have to rely on a softphone after all. :S

To adjust the alternative buttons click the “phone” tab and set each line to disabled, or name them as alternative lines as Louis does in this youtube video.

Now I just need to setup user 1000 since the 7925 was so nice to not use SIP, like at all SCCP only, no thanks for now. So, in today’s more modern times, I’ll just use a softphone. I decided to play with linphone on my andriod phone.

Open it up and four options appeared create account, use linphone account, use sip account, or fetch remote configuration… Use SIP account.

username: 1000
password: AsSetInXLM
Domain: IPofFreeSwitch
Name: Optional

And it connected!

OK so now out setup looks like this:

Now we have the very basics to make out first call: 2 Users and 2 Registered phones. So from the SPA525g, I dialed 1000, and sure enough my linphone rang, picked it up and had my first self configured SIP call. It was the usual self mocking type comments back and forth. After hanging up there was some feeling of accomplishment. But no time to stop here… there’s more fun to be had!

Unfortunately I was unable to make a call from linphone to the SPA phone cause as others have mentioned in the comments for some reason it auto adding +1 in front of all numbers dialed and it won’t simply ring ext 1001.

Important Tid Bits

  • Log directory – /var/log/freeswitch
  • Configuration directory – /etc/freeswitch
  • Database directory – /var/lib/freeswitch/db
    • Hosts SQLite databases
    • SQLite is the default database, many are supported
  • Daemon is configured via systemd
 sudo systemctl start freeswitch [or] service freeswitch start|stop|status
  • Administration – make yourself part of the freeswitch group
sudo usermod -aG freeswitch useraccount

Invaluable tools for administration – fs_cli (included in freeswitch)

let’s also install sngrep (this will come in handy later)

sudo apt install sngrep

FreeSwitch Configs

  • /etc/freeswitch/freeswitch.xml
    • This is the “point of entry” for configuration
    • It includes /etc/freeswitch/vars.xml, and does fileglob-includes for other important bits
      • autoload_configs/*.xml
        • This is where module configurations live (e.g. database connectivity, SIP stack, more)
      • dialplan/*.xml
        • This is where dialplans live (e.g. how do you dial out, IVRs, etc)
      • directory/*.xml
        • This is where user provisioning lives by default
  • /etc/freeswitch/vars.xml
    • This is where the “preprocessor variables” and generally very important variables live
    • You can think of it as “settings that you can’t change at runtime”
    • Includes all your favourite hits, such as:
      • <X-PRE-PROCESS cmd=”set” data=”default_password=1234″/>
        • Change this ASAP!
      • <X-PRE-PROCESS cmd=”stun-set” data=”external_rtp_ip=stun:stun.freeswitch.org”/>
      • <X-PRE-PROCESS cmd=”stun-set” data=”external_sip_ip=stun:stun.freeswitch.org”/>

*The “stun” entries are for NAT traversal; if you’re not behind a NAT device, you can change these to “host:your.domain” or your IP address.

Since our FreeSwitch is behind a NAT as shown in the first topology picture, I left these fields defaulted.

  • /etc/freeswitch/vars.xml
    • Let’s change our domain:
      • <X-PRE-PROCESS cmd=”set” data=”domain=$${local_ip_v4}”/>
    • Other notable entries:
      • <X-PRE-PROCESS cmd=”set” data=”internal_sip_port=5060″/>
        • SIP phones will register to your server on this port
      • <X-PRE-PROCESS cmd=”set” data=”external_sip_port=5080″/>
        • Calls will come from your ITSP on this port

At this point in his slides he goes on about making an external call, while I do plan on getting to that, I needed a VoIP provider so I’m currently working on getting a VoIP provider setup. In the meantime…

Voice MaiL

I sure enough left a phone ringing, for a good amount a rings it seemed and I was automatically transferred to a user voice mail, amazingly no configuration was required.

The softphone (Linphone) also didn’t seem to have an indicator for such a thing and after a bit of da googling, I found you simply dial *98.

On the SPA525g first time pressing the mail icon will ask you to enter the voicemail number, which I entered incorrectly and had to find this guide to help me figure how to change it.

Setup -> User Preferences -> Call Preferences -> VoiceMail

Although I was able to listen to the message I found I would always get cut off at 30 seconds.

Nope Any call gets cut off after 30 seconds… I’m about to give up on this shit…

Troubleshooting, Yay!

I did manage to get a bit of help from my buddy Troy and a nice user on the FreeSwitch channel on IRC in #FreeNode

We used sngrep and realized that I was not getting a ACK message from the phone.

As you can see no ACK….

Cyrillax from IRC mentioned enabling advanced debug…

sofia loglevel all 9
sofia global siptrace on

This will output a lot to the screen, so if you need to backscroll and are using putty ensure you add plenty of backscroll lines the default is 200, and that is not enough..

Checking the debug logs we can see the contact info is not what we wanted, the phone is trying to connect to the FreeSwitch via the public IP address:

Now on the SPA525g we entered Advanced config area opening up additional configurations and told the phone to use the outbound proxy after defining it (with the IP address of the FreeSwitch) which worked and we had calls with the IVR last as long as required. I’m not sure if this will suffice when it comes times for outbound calls, but well cover that when we get there. lol I’ve been saying that a lot.

I still wasn’t sure if the additional proxy configs was the right solution to the problem, although it did resolve the problem and acks were sent from the phone directly back to freeswitch. However every softphone I setup even after setting freeswitch to the proxy IP it wouldn’t work and I’d see the SDP sent with the Public contact in the field every time…

no matter how I configured the FreeSwitch XML config files I couldn’t seem to get it to provide the contact of the private IP not the public one, which I kept reading and hearing that’s normally what you want. I couldn’t see these requests for traffic in my Monitor tab of the Palo Alto firewall, so I thought it was a dud or wasn’t happening, but decided to create a U-Turn NAT rule anyway.

after committing I finally got an ACK! hahah from the firewall itself, kind of as expected since it TCP based, in this case and required to completed TCP’s 3 way handshake.

The diagram looks like this now:

Now things work, except for some reason I can’t call the softphone from the spa525g. But the Softphone can ring the SPA525g just fine…. ughhh my ignorance is causing gremlins! OK everyone can call 5000, and voicemail, but noone can call the softphone @ 1000. I’ll figure this one out tomorrow.

I decided to see if this was the problem, and reverted the outbound proxy settings I had added to the SPA525g. and sure enough go multiple SDP with no ACK, this time it was cause they were attempting to negotiate via UDP not TCP has my rule above I created for TCP only… OK let’s duplicate the rule and also allow UDP. Since it now is using UDP for the SDP and I did not define that port in my UTurn NAT rule, I”ll create another one for UDP but without source NAT translation… so it’ll look like this:

The rule looks like this now:

and after committing we get an ACK from the phone directly, without configuring an outbound proxy setting on the phone. 😀

Sure enough, on the SPA525g, everything works, calls to the 5000 built in IVR, VoiceMail, the works. Now lets try the softphone again… nope….

OK well I’m not sure if it’s the VIA field causing me grief or the fact that calls being made when routing from the FreeSwitch keep saying from 1001@freeswitch instead of the users making the call 1000. I removed the CIDR from users 1001.xml and copied it, changed the password, updated the config with:

fs_cli -x "reloadxml"

And oddly enough I was finally able to call the softphone on ext 1000. There was a long delay before the ringing started but it worked this time?! like what?

Sure enough I can call both ways now, but when I call 1001 from the softphone (1000) it rings right away, if I call 1000 from the SPAphone (1001) there’s a delay before the ringing starts. I’m not sure if this is some limitation of the app I’m using. I also have no idea how the heck making that change made the calls start working…

after creating two new users (copied 1001.xml and changed all 1001 to 1002 and 1003 respectively). Now calls going both ways are instant and all phones soft and SPA are working 100%.

I stand corrected… calling ext 5000 give me now a busy signal… this is starting to really annoy me…

Oh wait… right I changed 1001.xml with random 1005 numbers….

once I reverted this back to default as pictured at the top of this blog post, ext 5000 started working again… Not sure why this is but I guess it might be time to check out the dial plans?

30 Second Cut Off

Check to make sure the FreeSwitch Server is getting the required ACK. See Above for example.

Call Connects but No Audio with Direct SIP Routing

Check your internet connection NAT rule for the FreeSwitch server, ensure it is a Static NAT, not Dynamic IP and Port.

10 Second Delay In Call Answer

I searched this one up the other day, and I’ve heard it could be DNS (check you /etc/resolve.conf) mine was good. Heard it was due to STUN people set there STUN servers blank, this however will have consequences on the SDP contact information, so I wouldn’t recommend this, but it has been mentioned. In my case it was all of a sudden deliberate sleep execution due to not having changed the default password in vars.xml.

So yeah…. make sure you change the default password. then reloadxml in fs_cli.

Dial Plans and Phone Numbers

Different Dial Plan Directories

You may have noticed we have (used for internal phones)

 /etc/freeswitch/dialplan/default/

as well as (used by les.net dialing in)

 /etc/freeswitch/dialplan/public/

OK…. now we finally got past all the lower layer technical hurdles we can finally get to configuring the application itself. However we need to … collaborate with external sources. Now for me I’m lucky and have a local VoIP provider that is small in size but very technically aware, and much like Troy’s slide I use the same provider. Les.net

I tried to setup an account with them anonymously but that didn’t work as I had to call in as my account got suspended with fake info… Whomp wommmm womomomo.

So after I got my account verified, clicking on Order DID, pick the area, the area-code and any other information and the order details pop up (slide 37):

hahahaha, it’s cheaper for me to order a number for Fargo, ND then it is for me to order a local Winnipeg number… hahahah ahhhh… btw I am not using that number, I’ll still with the free DID for now, anyway…

The point is now we should have the basics in place to get FreeSwitch server registered with an external VoIP provider so we can make calls to the, sweet, sweet, candy… I mean public phone system. Sooo we are working on this:

As you can see the SIP/SDP/RTC/RTCP arrow is both ways, so if the les.net proxy send UDP based packets at me, the Palo Alto Firewalls will not know what to do with them, and drop em like they’re hot, drop em like they’re hot… When the Bi…. whoops going off track anyway, let’s create some rules to allow connection from our Internet telephony service provider (ITSP).

These details should be provided to you by your ITSP.

I thought about it a bit and did not create and open bi-directional NAT rule cause I’m sure my ITSP doesn’t want DNS and alternative requests from my freeswitch, so instead I created an open one way NAT rule that says anything from LesNet SIP proxy’s send it to my Freeswitch, in hopes those proxy’s are also setup to send only what they need to the right place. I still need a security rule though to make this work. So again I’ll leave it open, monitor the traffic and restrict the application or service ports accordingly.

Now that we got the firewall out of the way let’s go configure the FreeSwitch server.

/etc/freeswitch/sip_profiles/external/

This directory is for integrating with upstream providers. You can have multiple ITSP gateways. These handles incoming SIP traffic on port 5080
(Which we have our NAT and SEC rule so this should be good to go now)
Example: you could register a DID for multiple provinces, and have each trunk as it’s own gateway.
/etc/freeswitch/sip_profiles/internal.xml
This configures your internal profile (port 5060) for accepting connections from SIP phones (Which we already went through the nitty gritty above)

Let’s create a file in this directory, lesnet.xml

cp /etc/freeswitch/sip_profiles/external/example.xml /etc/freeswitch/sip_profiles/external/lesnet.xml

Use the username, password, and proxy provided by lesnet’s login page(To do this on LesNet have to create a new SIP Peer / Trunk, then click the edit button on it, this will present the required details to enter into the xml file.)

New profiles can be loaded at runtime

fs_cli
sofia profile external rescan
sofia status gateway proxy.sip.les.net

Note – “sofia” is the name of the SIP stack used by freeswitch.

WOW! it worked!

Since this was a connection from freeswitch to lesnet I didn’t see it hit my newly created rule instead it used my default home network outbound rule which was allowed.

My excitement was again short lived as I hit another road block (story of my life). Turns out I kept seeing repeated Registrations and 401 responses. I wasn’t sure of this and made a change to my external gateway…

nano /etc/freeswitch/sip_profiles/external/lesnet.xml

sofia profile external restart reloadxml

So to get out bound to even show up on the lesnet side some changes were required.

nano /etc/freeswitch/vars.xml
 <X-PRE-PROCESS cmd="set" data="default_provider=proxy3.sip.les.net"/>

then again some reloadxml

fs_cli -x "reloadxml"

now when we make calls it’s bust but at least they show on the call logs on the ITSP portal.

Incoming Calls

Now for incoming calls, after you verify a stable connection with the ITSP Gateway/proxy, and see it their online portal, you may have to map a number to a DID Peer/Trunk, In this case I saw my registered FreeSwitch as SIP Peer 79908, then under “Your DIDs” have to click on the number you wish to route, and select the end SIP peer to route those calls to, in my case SIP Peer 79908.

At this point you should be able to see the calls come in on the ITSP call logs and the FreeSwitch via sngrep, but it won’t be routed anywhere according to FreeSwitch’s dial plan so…

nano /etc/freeswitch/dialplan/public/1204666xxxx.xml

Now I don’t think you have to name it this way, pretty sure you can name it differently but this is for simplicity for now. and fill it with:

This should be all that’s required, just do another reloadxml, and dial the number.

Outbound Calls

Now with the current Dial-Plan that’s defaulted 01_example.com.xml it’s using the gateway variable we defined in vars.xml so our only outbound proxy at this point. Since I was able to see the calls hitting the les.net portal but getting denied I decided to give les.net a call to see if maybe they had an idea why.

When checking my SIP peer trunk on the portal which was my FreeSwitch it was registering every 20-30 minutes, it was suggested to drop it to between 60-90 seconds.

So in the gateway settings:

nano /etc/freeswitch/sip-profiles/external/lesnet.xml

thx

sofia profile external restart reloadxml

Turns out that wasn’t the case, I had a hunch the problem was the fact the source was 000000000 as your can see:

so I quickly googled this to see if I could find something.. I found this…

“dial and bgdial

If the caller id values are not set, the variables in conference.conf.xml will be used. Specifically, the value for caller-id-number will be used for the number and the value for caller-id-name will be used for the name.

If the conference will be dynamically created as a result of this api call (ie this will be the first participant in the conference) – and the caller id name and number is not provided in the api call – the number and name will be “00000000” and “FreeSWITCH”. This appears to be unaffected by the variables in conference.conf.xml.”

Ohhhhhhhh… ok so if I set the call outbound number in the user file…

This works for the single user, to make it more of a NAT like you do with a single public IP address and want to share the internet, you set this variable in the vars.xml file.

and sure enough :D…

Yes!! hahah finally.

That’s it for now! Next round I’ll cover IVRs and all the other fun stuff. This is just the basics. and even then doesn’t cover it very well, just enough to get it all to work. I also noticed that I didn’t have to the NAT rules or the security rules so just the basic NAT is required for FreeNAS and the phones I guess… hahaha

 

Posted on

Even More PowerShell Fun

The Story

It’s another day, and we all know what that means… yes, another blog post, and even more PowerShell! Can you feel all the power!?!?!

This time it came down to the storage size of my Exchange servers C:\ which turns out to be due to Logs. Logs are great, and best practice is to only clear them if you have a backup copy. Often is the case that logs can be truncated after a backup via VSS by many backup solutions however in my case I could and probably should get that validated with Veeam (as I can’t seem to get that working ‘out of the box’) at the moment. So instead I wanted to know what was “usually” done server side even if someone was not implementing a backup solution.

Source: https://social.technet.microsoft.com/wiki/contents/articles/31117.exchange-201320162019-logging-clear-out-the-log-files.aspx

Neat, but the script is just alright, good for them doing what they want and that’s running it as a scheduled task. Not my goal, but a great source and starting point… let’s have some fun and give this script some roids, much like my last one… I’ll give this a home on GitHub.

Things learned…

  1. Working with the Registry
  2. Determining if Elevated (This is great and I may have a solution to the conundrum in my previous PowerShell post)
  3. Getting a Number, and validating it
  4. Validating Objects by Type
  5. Getting Folder Sizes

Check out my script for all the fun coding bits. I’m a bit tired now as it’s getting late so not much blogging, all more coding. 🙂

ErrorAction Stop Not Stopping Script

Quick Educational note (Source)

$ETLLogKey2 = 'HKLM:\SOFTWARE\Microsoft\Search Foundation for Exchange\Diagnostics'
try{Get-ItemProperty -Path $ETLLogKey2 -ErrorAction Stop}
catch{Write-Host "No Key"}
Write-Host "This should not hit"

Produces:

Well poop… The catch block was triggered but the script did not stop…

Oddly, changing to Throw, which is ugly does make the script stop…

$ETLLogKey2 = 'HKLM:\SOFTWARE\Microsoft\Search Foundation for Exchange\Diagnostics'
try{Get-ItemProperty -Path $ETLLogKey2 -ErrorAction Stop}
catch{throw "No Key"}
Write-Host "This should not hit"

Nice it worked this time, but it’s ugly…

Write-Error is just as ugly, but doesn’t stop the script?

$ETLLogKey2 = 'HKLM:\SOFTWARE\Microsoft\Search Foundation for Exchange\Diagnostics'
try{Get-ItemProperty -Path $ETLLogKey2 -ErrorAction Stop}
catch{Write-Error "No Key"}
Write-Host "This should not hit"

Produces:

Yet if I follow Sages answer in the source, and do a script variable for the stop action it then works???!?!

$ErrorActionPreference = [System.Management.Automation.ActionPreference]::Stop
$ETLLogKey2 = 'HKLM:\SOFTWARE\Microsoft\Search Foundation for Exchange\Diagnostics'
try{Get-ItemProperty -Path $ETLLogKey2}
catch{Write-Error "No Key"}
Write-Host "This should not hit"

Those are really weird results, but all still ugly… so it seems even though -ErrorAction Stop causes a non-terminating error to be treated as a terminating error, but depending on what you do in the catch block determines if there’s a break/exit event being done. In my case to have things look nice and actually stop the script I have to do the follow.

$ETLLogKey2 = 'HKLM:\SOFTWARE\Microsoft\Search Foundation for Exchange\Diagnostics'
try{Get-ItemProperty -Path $ETLLogKey2 -ErrorAction Stop}
catch{Write-host "No Key";break}
Write-Host "This should not hit"

Which finally produced the output I wanted. (I could have also used exit in place of break)

Finally!

Validating URLs:

Source:

Good but broken due to the match on the scheme extension:

function isURI($address) {
	($address -as [System.URI]).AbsoluteURI -ne $null
}

function isURIWeb($address) {
	$uri = $address -as [System.URI]
	$uri.AbsoluteURI -ne $null -and $uri.Scheme -match '[http|https]'
}


isURI('http://www.powershell.com')
isURI('test')
isURI($null)
isURI('zzz://zumsel.zum')

"-" * 50

isURIWeb('http://www.powershell.com')
isURIWeb('test')
isURIWeb($null)
isURIWeb('zzz://zumsel.zum')
isURIWeb('hp:') #Return True

 

Better results with:

function isURI($address) {
($address -as [System.URI]).AbsoluteURI -ne $null
}

function isURIWeb($address) {
$uri = $address -as [System.URI]
$uri.AbsoluteURI -ne $null -and $uri.Scheme -match "http|https"
}


isURI('http://www.powershell.com')
isURI('test')
isURI($null)
isURI('zzz://zumsel.zum')

"-" * 50

isURIWeb('http://www.powershell.com')
isURIWeb('test')
isURIWeb($null)
isURIWeb('zzz://zumsel.zum')
isURIWeb('hp:') #Return True
Posted on

Using A USB Device as a Datastore on VMware ESXi

USB datastore

Attach USB device in Windows -> DiskPart -> Select Disk -> Clean

To see USB device on host, stop arbitrator service, save to config, reboot

Now the hard parts, normally even following guide to see USB stick on host mount 4 gig volume, I had 16 gig cruiser to use.

Source

Perform a lspci -v to get all the USB UHCI and EHCI controllers to show up.
This shows up for example as:

DEVICE=/dev/disks/t10.JMicron_USB_to_ATA2FATAPI_Bridge
partedUtil mklabel ${DEVICE} msdos
END_SECTOR=$(eval expr $(partedUtil getptbl ${DEVICE} | tail -1 | awk '{print $1 " \\* " $2 " \\* " $3}') - 1)
/sbin/partedUtil "setptbl" "${DEVICE}" "gpt" "1 2048 ${END_SECTOR} AA31E02A400F11DB9590000C2911D1B8 0"
/sbin/vmkfstools -C vmfs5 -b 1m -S $(hostname -s)-local-datastore ${DEVICE}:1

That easy 😉

Posted on

Outlook and the Cache Mode

The Story

Wouldn’t be another post without another day of annoyances… yup, just another day. So recently it’s been reported that when users are working remotely their outlook decides not to open….

Now since they are remote, I have noticed this only happens if the Outlook client is configured in “Online mode”, instead of “Exchange Cached Mode”… see this MS Docs for more information on the different types and when to utilize each mode.

Originally I configured Online Mode, as most users are locally available in the work network and this is not a major problem. Also when using “Cache mode” not all emails show up in Outlook right away, specially if using folders, there generally shows a link “Click here to view more on Microsoft Exchange” this means items that were not cached (depending on the cache time slider this may vary, I choose 12 months), view this support if you can’t see the link in cache mode enabled.

I used to have an issue with this setting, user reported items wouldn’t load even after clicking the link, however I haven’t seen this to be an issue anymore, so I recommend to enable Outlook Cache mode unless you fall under the other points in MS’s Doc linked above.

Finding Users That Are Not Using Cache Mode

So I was now on a new mission… “How do I know who’s not using Cache mode on outlook?” And this is were the rabbit hole began….

First result, same question…

Craig Harts reply…

his or response seem like it’s rather easy until…

wtf is this…? Depreciated?!?! C’mon, but there’s an alternative now that’s better right? No…. Thanks MS… seriously… thanks…

So this older post goes over alternatives, not because they didn’t have access to the cmdlet above, but simply cause they didn’t trust the results.. huh…

Problem is this seem to be reg keys used by older outlook, and new outlook seems to use alternative keys… This sure is fun! However, thanks to others that blog and code in their spare time as well, in this case thanks to Jose Espitia much like in my last blog post this is a great start, but usual me, I don’t like expecting anyone to change the source code. In this case you have to provide a file with a computer list, output path, but it’s hard coded…. OK you know what that means! Yeah I usually would create a new GitHub Repo but first…

Much like the older post mentioned they choose to target end user directly to get the most “accurate data”. This however assumes four things:

  1. That Exchange admins are workstation admins, and have elevated rights on all machines.
  2. That all firewalls are configured and permissions to allow remote querying.
  3. That the user’s are in fact online at the time the query is made.
  4. That RemoteRegistry Service is started and running on end machines.

In my Case it was not and disabled on all machines, I didn’t feel it was beneficial enough to introduce a risk for the simple sake of determining a users connection mode on Outlook.

So I decided to go back to Option A from the old original post, parsing all the RPC Access logs… Using this an alternative reference.

Well I gave it a try and looking through all th elogs there was no reference to “Classic” so I guess that’s no longer valid option either.

Looks like I’m stuck on this one, I can’t seem to find a valid way to find this information out server side with MS removal of the Get-LoggingStatistics cmdlet. And attempting to query all ends users has to many restrictions/hoops that I do not wish to implement. In this case I have to simply go around to all users machines, or wait for the to complain when they work remotely.

Thanks Microsoft I really love what your doing for SysAdmins. Taking away mark and putting him on as COO of Azure so all our beautiful tools from SysInternals are now just the way they are, and new tools, don’t need em right, just buy your cloud subscriptions and who needs SysAdmins… 😛

Anyway… that’s it for today. Sorry no advanced scripts form this post, just use Jose’s script if you wish to query end users machines. Just ensure you have RemoteRegistry service running on all end users machines, and not blocking it in the firewall as well. Hoops I have no interest in jumping through.

Cheers!

Posted on

Powershell fun

PowerShell

Finding specific files

The Folder path in which you need to search
$Path = ‘C:\folder\path’

The Regular Expression you like to match
$RegEx = ‘[^-\w\.]’

The CSV File name
$OutCSV=’C:\temp\fileResults.csv’

Get-ChildItem -Path $Path -Recurse | Where-Object -FilterScript { $_.Name -match $RegEx } | Export-Csv -Path $OutCSV -NoTypeInformation

 

And YA! files modified in the last 24 hrs…

Get-ChildItem -Path $Path -Recurse | Where-Object -FilterScript { ((get-date) - $_.LastWriteTime).totalhours -le 24 } | Export-Csv -Path $OutCSV -NoTypeInformation

Finding Objects Properties

I used to use

$Object | FT *

to try and find properties to use, but easier is

$Object | get-Member

So much easier to see all an objects properties, and methods… and whatever 😀

There are obviously endless things you can do with powershell, with a syntax farrrrrrrrr more intuitive then BATCH ever was. That is really thanks to a lot of object operations relying on non other then regular expressions AKA RegEx.

RegEx is powerfull but also hard to grasp at times. Which funny enough has caused this blogger an unknown grief :P, though someone suggests not to use $ in between double quotes but for single quotes.. but it’s not “just that easy” as understanding the difference between using single quotes and double quotes is a big thing in PowerShell and understanding this as PowerShell as much as the RegEx underneith has it’s own special synatx on certain things and certain characters, so not only does PowerShell interpreter have it’s own parser with escape characters… So does RegEx so you have to know the info in the above link, as well as understand RexEx’s syntax, special characters, and escape characters specially when using operators like -like -match -contains or even -AllMatches…

Finding Files With Specific Characters

Outside of setting variables this really comes down to one line in the alternative bloggers script… line 8…

$item.Name | Select-String -AllMatches $UnsupportedChars

Nice, so simple, but as mentioned this breaks if you attempt just specific special characters, such as $ or ^ as these are special characters to RegEx and need to otherwise be escaped to be taken literally.

Now the script in and of itself is simple enough to accomplish a simple task.. to explain in better detail what I mean… here’s my list of files, NTFS accepts more strange characters then I first though, so users can really be creative…

clearly much more complex, so if I run his script what happens….

Now I commented out the line that actually renames the files, so you know, I don’t have to rename them to reset my test. That’s good, but even though he added the money sign in his blog to the “$UnsupportedChars” var, he didn’t actually add it in the final script, and what happens if you do?

ok so in this case, the -AllMatches worked without issues I suppose due to changing to single quotes. but what if we need to change it’s character, let’s copy one of his ifs that changes it and see how it responds…

The results…

hahahha that’s a lot of money, clearly the RegEx got confused when we specified $ in the -Replace operations… so as the source RegEx escape characters says, escape it with \…. lets do that…

Awesome it worked… but I don’t like edited a script every time I need a slight change in my search criteria…. let’s take his script… and well completely redesign it as if it were on Steroids!!! Functions! – to reuse reusable code with returns! Write-Hosts and Read-Hosts for user interaction and input, and heck why not options on results! Export, List, rename sounds good! This along with some dependency checking! (Does the path exist? Do you have rights to modify?)… oh yeah and confirmation of action! 😛

This beast will need it’s own GitHub Repo… so… here it is!

OK… now that I’m actually writing this blog actively while I’m polishing my script.. I noticed my export section had a Try, Catch for the creation of the CSV file. Simple but this, in itself, is not actually checking if I have permission… it’s simply checking to see if the I/O operation is actually able to succeed… it may fail due to other reasons (store space for example). So how do I check to see if I actually have permission before continuing? Well check your group membership (along with your own SID) and check the files/folders Access Control Lists (ACLs), obviously.

So how do I get a workaround based machines local user group membership?

Googling I can find easy how to do this for a domain joined machine, which is fantastic if I wanted to validate the path on a network share where the ACE’s and ACL’s are all managed by domain SIDs. But this test and script is all on my “workgroup” machine… and finally after enough searching the holy grail!

Wasn’t done directly by Ed Wilson but close enough, it gave the the code I needed to check the members of the local admin group, but what I really need is simply to validate read/write permissions.. So I found this… and sadly it doesn’t understand the fully distinguished user name when using whoami as the user variable… and it can’t sense inherited permissions… only once I granted my akadmin direct read and execute permissions would the code work…

Like ughhhhhhhhhhh………

I’m gonna skip this in my script for now… rather work on getting export to work properly, and actually getting the rename code in when saying yes. :O.. will update this post when that’s done.

… and yet in my drunken’ stuper I decide to keep trying…

I think the only way is only to iterate through the local groups and verify users.. nope leaving this again, unless a better way comes about.

OK.. so I felt I was on to something…

figured there’s got to be an easy way to do the same cmd as whoami /groups in PowerShell right… wtf is going on here?!?! I thought PowerShell was the easier scripting language… man this is crazy. and this

Not even brah… THIS! This is getting close!

This is gon’ be fun! (Great Source)

$folder = "c:\temp"
$CP = New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())
$UGM = $CP.Identities.Groups | ?{$_.AccountDomainSid -ne $null}
$UGM = $UGM + $CP.Identities.User
$TPP = (Get-Acl $folder).Access
foreach ($P in $TPP.IdentityReference.Translate([System.Security.Principal.SecurityIdentifier]))
{
foreach ($UP in $UGM){if($UP -eq $P){Write-Host "We got permissions here!`n";$TPP | ?{$_.IdentityReference.Translate([System.Security.Principal.SecurityIdentifier]) -match $P}}}
}

Testing this out on my the account I added to a group worked just fine, but on my main machine it failed on me…

Poop, what happened… turned out my code filters the nitty gritty built in groups. By default all users are a part of these and aren’t generally used to define permissions (but can be, E.G Administrators). Since there was only one group my account was a member of the PS Object “$UGM” was not of a multi-value object aka an array object and doesn’t have an addition type method to complete the addition of the user SID, to the what should be array, as I have not defined it as such. So this is a good example to usually always define your variable types or fault to the problems they can inhibit. Also I was stunned I was not able to see the group SID when I called for all local groups, no matter how I called them, so doing the magic translate trick as the source above…

What the heck… None … didn’t even know such a group existed…

Code fixed by doing the following:

$folder = "c:\temp"

$CP = New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())
$UGM = @($CP.Identities.Groups | ?{$_.AccountDomainSid -ne $null})
$UGM = $UGM + $CP.Identities.User
$TPP = (Get-Acl $folder).Access 
foreach ($P in $TPP.IdentityReference.Translate([System.Security.Principal.SecurityIdentifier])) 
{
 foreach ($UP in $UGM){if($UP -eq $P){Write-Host "We got permissions here!`n";$TPP | ?{$_.IdentityReference.Translate([System.Security.Principal.SecurityIdentifier]) -match $P}}} 
}

After enough beers I got it to even let the user know if they are in the admins group but not elevated 🙂

$folder = "c:\temp\None"

$CP = New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())
$UGM = @($CP.Identities.Groups | ?{$_.AccountDomainSid -ne $null})
$UGM = $UGM + $CP.Identities.User
if(([Security.Principal.WindowsPrincipal] [Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator")){$UGM = $UGM + ($CP.Identities.Groups | ?{$_.Value -eq "S-1-5-32-544"})}

$TPP = (Get-Acl $folder).Access

foreach ($P in $TPP.IdentityReference.Translate([System.Security.Principal.SecurityIdentifier]))
{
    foreach ($UP in $UGM){if($UP -eq $P){Write-Host "We got permissions here!";$TPP | ?{$_.IdentityReference.Translate([System.Security.Principal.SecurityIdentifier]) -match $P}}}
}

foreach($pu in (Get-LocalGroup Administrators | Get-LocalGroupMember).SID){if($pu -eq $CP.Identities.User -And !([Security.Principal.WindowsPrincipal] [Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator")){Write-Host "You are an admin, but you are not elevated..."}}

Fixing the CSV Export

So far I have not go the permission checking working just yet. I have however managed to fix the export-csv issue.. old export…

Yeah that System.Object[] isn’t very helpful..

based on this old code:

$shit | Select-Object Name, Directory, MatchedValue| Export-Csv -Path $CSVFile -NoTypeInformation

I was trying all these weird things to try n resolve it (bad ideas)…

a lot of extra code for not good results, I figured there had to be a better way… and sure enough there was… Source

which left me with this code:

 $BadFiles | Select-Object Name, Directory, @{Expression={$_.MatchedValue -join ';'}} | Export-Csv -Path $CSVFile -NoTypeInformation

which now provides this output:

Nice! Until the moment I realized, can user use ; in file names… Ahhh they can! shoot! So now my test files grows larger, but my script help up with very lil modification! first to add it in the list of searchable characters:

function GetChars()
{
    Write-host "Characters to search for !$'&{^,}~#%][);( " -ForegroundColor Magenta -NoNewline
    $UnsupportedChars = Read-Host
    if ( !$UnsupportedChars ){ $UnsupportedChars = "\]!$'&{^,+}~#%[);(" }
    return $UnsupportedChars
}

That was it! the list output is fine:

but the export not so much as I was using ; to join those items…

K well users can use a \ so let’s see if we can use that in the export, again easy code change..

$BadFiles | Select-Object Name, Directory, @{Expression={$_.MatchedValue -join ';'}} | Export-Csv -Path $CSVFile -NoTypeInformation

Let’s test…

Nice, also added =, and let’s also test for @…

easy got em… until I realized you can name a file with a backtack `

and if I search for it on it’s own no problem…

but if I try to extend the default list of characters, I can’t seem to get it to work, even if I specify an escape \ in my list… Woah! I randomly got it!

function GetChars()
{
    Write-host "Characters to search for !$'&{^,}~#%]`[)@=;( " -ForegroundColor Magenta -NoNewline
    $UnsupportedChars = Read-Host
    if ( !$UnsupportedChars ){ $UnsupportedChars = "\]!$'&{^,+}~#%[)@=;(``" }
    return $UnsupportedChars
}

Double backticks worked, what a random thing to try and it actually worked…

That’s it for tonight! I got pretty much every special character covered. 🙂

Next time I want to cover bulk confirmation with a report, since confirming every one can be annoying, but better to have confirmation then not!

PIssssss!!!!! Source of problem

Oi! I DID IT!!! YES!

From this…

        if(confirm $turd.name $turd.NameToChange)
        {
            Try { Rename-Item $turd.name -NewName ($turd.NameToChange) }
            Catch { Centeralize "Unable to change file's name"$turd.name "Red"}
        }
        else
        {
            Write-Host "Maybe Another time..."
        }

to this (after exporting to it’s own function though)…

        if(confirm $turd.name $turd.NameToChange)
        {
            
            $ThatOldName = $turd.FullName
            if (($ThatOldName).ToCharArray() -ccontains "[") {Write-host "We found a bad char";$ThatOldName -replace "\[", "`["}
            Try { Rename-Item -literalPath $ThatOldName -NewName ($turd.NameToChange) -ErrorAction Stop }
            Catch [System.UnauthorizedAccessException]
            {
                Centeralize "You do not have permission to change this file's name" "Red"
            }
            Catch 
            { 
                $ErMsg = "Unable to change file's name "+$turd.name
                Centeralize $ErMsg "Red"
                $ErrMsg = $_.Exception.Message
                Centeralize $ErrMsg "Red"
            }
        }
        else
        {
            Write-Host "Maybe Another time..."
        }
    }

I can’t believe it worked! wooo results from this:

To this!:

I can’t do much about files that already exists, besides people mentioing way to over-write existing files which I’m not coding for, nope sorry.

but so far so good. This script is awesome!

Today I discovered that Try-Catch only works on terminating exceptions…

Powershell ErrorAction

Sigh… This means I have to do additional coding if I want the script to get all items from main folder, even if there’s a permission restriction on a child folder.

Life’s rough!

Legal Information Proudly powered by WordPress