Palo Alto VPN (GlobalProtect)
Part 5 – Rules, Testing, Troubleshooting

Intro

In this 5 Part series I covered all the requirements to configure Palo Alto Network’s GlobalProtect VPN:

1) Authentication, Auth Profiles and testing them.

2) Certificates, Cert Profiles, SSL/TLS Profiles and creating them.

3) Portals, what they do and how to configure them.

4) Gateways, what they do and how to configure them.

This part will cover the security rule required, and a little troubleshooting steps along the way.

Things not Covered

I didn’t cover creating DNS records, as again, these come down to your own DNS provider and whatever tools and portals they offer to manage those.

I don’t cover configuring the interfaces (public facing or internal), I don’t cover the virtual router and routes. All these are assumed to be handled by the administrator reading these guides.

I don’t cover installing the client software, if you have the certificates installed on the client devices (Required), it’s simply navigating to the portal address with a supported browser and downloading the installation packages (.exe for windows).

For giggles, I tested navigating my portal from my phone, it did prompt me for my certificate (the VPN was working well) yet after selecting my certificate I got a connection reset error on my browser and checking the Palo Alto Firewall logs (Monitor tab -> traffic) I indeed saw the Deny traffic and action reset-both action… why this is, even though the application was identified correctly as web-browsing and that was enabled in the rule, it wasn’t being allowed by my rule and instead was being denied by my deny all rule. I”m not sure exactly why this is, however I don’t have intentions of accessing my portal web page anytime soon, so for now I’ll ignore this as I use IPsec XAuth RSA on my android device.

I have also noticed that for some reason with Samsung Android I can’t seem to get this VPN setup to work, from quick google searches people seem to say it’s due to packet fragmentation somehow. I haven’t yet had the chance to look into the nitty gritty of this issue just yet, but when I do it will be it’s own blog post!

I also don’t cover installing the completed certificates onto end devices as again this comes down to the end devices being supported by the administrator configuring Global Protect and is outside the scope of this guide.

The Security Rule

As you can tell pretty simple, anyone from the internet (I could be connecting from anywhere, and my IP address changes on my phone all the time, random access points etc) to my public IP address which hosts my portal and gateway, and the required applications (IKE, ipsec-esp-udp, and the SSL and web-browsing) again I haven’t exactly figured out the portal web-page loading issue just yet.

 

*UPDATE* ensure to add panos-global-protect application type, else only X-auth RSA connection will succeeded, that does not rely on the Global Protect Portals.

Failure to add panos-global-protect applicatin results in end client getting “No Network Available” error on the Global Protect App.

My Phone Config

In my case I do run an Android phone, running : 8.0.0: Kernel 4.4.78

The OS is some H93320g couldn’t find much but this about it

For the most part I install both my Offline-Root-CA and my Sub-CA certificates on my phone. Which can be found under (General -> Lock Screen & Security -> Encryption & Credentials -> Trusted Credentials (Instead of CA’s who knows?) -> User (Both Should be listed here)

Then Installed the User certificate with the private key, which then shows up under (General -> Lock Screen & Security -> Encryption & Credentials -> User Credentials (Instead of User Certificates?)) The other annoying part is once you have the certificate installed, this area doesn’t allow you to see the certificate details, you can see them under the area mentioned above, but this area…. nope.. :@

Once the certificates are installed, it simply comes down to configuring the VPN settings. (Settings -> Network -> VPN -> BasicVPN -> Click the plus in the upper right hand corner. Then)

Name: Give it a meaningful name

Type: IPSec XAuth RSA

Server Address: The Address defined in Part 3 -> Agents -> External Gateways

IPSec User Cert: The User Certificate you installed and verified above

IPSec CA Certificate: Don’t verify server (Which is probably why I didn’t need the above server address in the gateway certs as a SAN)

IPSec Server Certificate: Receive from server

Then enter a username and password for a user you defined to be allowed per your Authentication Profile you created in Part 1.

You shouldn’t have to define the advanced settings as those should defined to the client from the gateway config we created in Part 4.

Summary

If done correctly you should have a successfully, you should be able to see all the parts play out in both the traffic logs, and the system logs…

System:

Traffic:

That is pretty much it, if you have a failed connection do the usual step by step troubleshooting starting with connectivity, you should be able to see the access attempt from the device in the traffic logs, if they are being blocked by rules, adjust them accordingly.

If you verified all other things, it maybe your chain, or you are enabling extra security like verifying the server certificate than you chain would have to be different then presented here, probably all certificate including the portal and gateway certs being signed by the sub CA completely, then all certs will be trusted by all devices. I’ll admit this isn’t the cleanest setup, but it’s the closest to a bare minimum install of Global Protect using your own internal PKI.

I hope this guide helps someone. 😀

Palo Alto VPN (GlobalProtect)
Part 2 – Certificates

Certificates

In my previous post I covered recovering a downed CA, cause it will be needed for this section of the GlobalProtect tutorial.

Step 1) Importing the CA Certs

We need to add all the CA certs that are involved in completing the chain, so this includes, the Offline-Root-Ca, as well as the Sub Ca.

Adding the Sub CA cert:

Device -> Certs -> Import -> Base64 cer file

Step 2) Generating a CSR

Generate a a Sub CA Key for the PA to handle the Gateway certs, afterwards generate a Gateway certificate as well.

Click generate:

Click Generate

export the CSR, for some reason the latest Chrome causes a constant refresh, argggg had to export the CSR via IE, gross….

Navigate to your CA’s signing Web page (the Sub CA in this case), open the CSR in notepad and paste the results, and select Sub CA for the template:

Then save as Base64 type cert, and import back into the PA firewall, if successful will look like this:

Also import Offline-root-ca cert to complete the chain

Step 3) Certificate Profiles

Alright time for Certificate Profiles

Add all the Certs

Step 4) SSL/TLS Profiles

Create a SSL/TLS Profile:

Name it whatever, pick TLS 1.2 as min and max, and select the PA Sub CA we created earlier.

Step 5) Create User Certificate

Step 5.1) Create Template on CA

Then under Cert Templates, right click it, and duplicate

5 Years, i don’t like doing this often

Signature and encryption, check off include symmetric allowed by subject, min key size of 2048 and key is exportable

Along with the default, check off MS RSA and AES, and RSA SChannel

Subject Name, Supply in the Request, it will complain about the security risk, accept them. (Normally you’d create the certificates at the client machines, but in this case I am doint it the “wrong way” by having a global user certificate)

Click Apply.

If you require additional permissions apply them now, by default domain admins have full control, and domain users have enroll rights.

Step 5.2) Generate User CSR

With the Template configured, lets create the User Cert for the VPN, in this case we generate the CSR on the PA, but since we made the key exportable, we can export the certificate with key to be installed on the end device (instead of the CSR being generated on the device and then signed, and the public key being installed on the portal, which is the right way… hopefully I can get that, but the toughest part is generating certificates on phones, have to learn each devices OS on how to do it)

On the PA Device, Certs, Generate

*NOTE* I noticed that with the latest Chrome that when you attempt to export any certificate it just seems to refresh the page, sadly the only work around I have is to use IE… Ugh….

Open the CSR in Notepad, navigate to your Sub CA’s certificate signing page, sign the certificate.

*Secrete enable remote management on IIS Core*

lol, I was wondering why i couldn’t see my Template in the web interface, so I looked up my own very old blog post (3rd one I believe) and I realized I forgot to publish it, like I did the Authentication Session Template. Durrrr, then it kept complaining about https for cert destro (makes sense) but since I had a core subca, I couldn’t connect to the IIS remotely, then I found this, saved my bacon, and followed this to enable HTTPS, Then finally…

then Import it on to the Firewall,

it should look like this

In the next section I’ll cover configuring the Portal and Gateway settings. 😀

vCenter Network Partitioned

Have you ever experienced a Network Partitioned warning in vSphere 5? Hopefully not, but if you find yourself with this warning in vSphere. Don’t panic its not as bad as it could have been in 4.x.

This literally just means that the host can not communicate with any of the designated VMK’s checked off for management traffic. In my case it happened after making network changes to my infrastructure. In this case I still had bonded links at my switches, but somehow the VMK load balancing algo had switched to “route based on originating port ID”, this load balance algo doesn’t work with bonded NICs, and needs to be “route based on IP hash”. My end goal was to get off bonded links for my host and use the default load balance algo that VMware uses, as this can be down with non stacked switches and can be done with minimal switching knowledge (in case others need to manage the system in the future).

It took me a little bit to catch the issue, cause the symptoms were that each host could ping any device in their respected management subnets but NOT the other host, flat /24 subnet too, really had me baffled. As I couldn’t vMotion in this state either, but lucky the VMs on each host remained active (as they have separate communication VMPGs on dedicated physical connections).

Once I caught the error, I was able to verify vMotion worked again. That’s all there is to it!

To Paraphrase to solution:

1) Check which VMKs have management checked off.
2) Check those vSwitches physical connections.
3) If multiple ports check configs on physical switch and load balance algo.
4) Google any errors along the way.
5) Check host to host communication by consoling into host and using vmkping.

Jan 2018 Update

I remember this…

Changing Network Location to Domain

Have you ever restored a VM? Have you done your DR testing by actually doing a full recovery with AD? Did you find you had a couple odd things occur after restore, such as not being able to RDP into your recovered server? Chances are your network profile has changed to public, instead of Domain. This in turn causes certain firewall rules to trigger.

I remember coming across this issue multiple times, especially when people usually want private instead of public and vice versa. So chances are you’ve come across this, telling you to use PowerShell cmdlet to change its setting, which to my guess makes a registry change. The other option they specified was to use the GUI.

Well I find changing local security policies and all that other stuff rather annoying. Soo after a bit more googling I found a really nice answer, which worked and was very simple to implement. Very nicely written and easy to follow by a Evan A Barr. You can view his site here.

To Paraphrase to solution Using Network Connection Properties:

0) by adding a DNS suffix so that NLA can properly locate the domain controller.
1) Go to Network Connections.
2) Go to the properties of of the network adapter in the wrong location.
3) Go to the properties for IPv4.
4) Click the "Advanced..." button.
5) Select the DNS tab.
6) Enter your domain name into the text box for "DNS suffix for this connection:".
7) Disable and then enable the connection to get NLA to re-identify the location.

Windows Shares over SSH tunnel

I am the worst at writing blogs. I seldem get excited enough to write anything. But today…. TODAY! I feel like this is going to be a good blog.

A fanastic blog… anyway, so I moved into a new place, but have my server still running at my old place I run a very lightwheight server from there.
pssssst, it’s really just a router but perfect for hosting a network shares, torrents, web servers (cough this page), ssh and smb (cough this as well)

If you haven’t heard about DDWRT, I’d suggest you check it out here

Anyway, while i use SSH tunnel to manage this router via CLI, I can always tunnel its web management interface port, to my local machine and manage it that way too.
Yes most changes does cause it do it a soft reboot and breaks the connnection, a simple reconnect after a couple minutes useally all it takes.
I figured I’d just forward the servers SMB port just like I do most of my other ports… to my dismay it didn’t work… so I decided to GOOGLE!

As it turns out, there is more tweaking required to do this that I first thought, like disabling the SMB service at start-up, and using a loopback interface..
If you have a Windows share server (SMB) at home and happened to have SSH for management also available, then check this link out!

Bye for now….

Jan 2018 Update

These are always neat tricks to keep in the back of your head, even if your playing around just for fun. I wouldn’t see the real world use for this type of hack today as everything is pretty much OpenVPN or some other VPN solution. Still love my SSH though.

Lucky the link is still active otherwise this post would be as useless as tits on a bull.

Feb 2019 Update

Mind Blow

This is a blog post from someone at the Dutch National Institute for Subatomic Physics… dude, that’s awesome!!!!!!