Deploying Pi-hole v6 Offline on Proxmox VE with a Palo Alto Firewall

Deploying Pi-hole v6 Offline on Proxmox VE with a Palo Alto Firewall

An architectural summary and step-by-step rebuilding guide for the containerized offline deployment.

🏗️ Technical Architecture Overview

    • Hypervisor: Proxmox VE (PVE)
    • Parent Compute Layer: Privileged Linux Container (LXC) running a minimal Debian 12 base template.
    • Network Blueprint: Bound to vmbr1, isolated natively via VLAN 7 (172.16.7.2/24), routing through gateway 172.16.7.1.
    • Runtime Engine: Decoupled, standalone static binary installation of Docker Engine v27.3.1.
    • Application Tenant: Pi-hole v6 running in native Host Networking Mode (--net=host).
    • Upstream Security: Border perimeter managed by a Palo Alto Networks (PAN) Layer 7 firewall with strict application checking and URL filtering.


🛑 Phase 1: Perimeter Firewall Provisioning (Palo Alto)

Because the Pi-hole requires internet access to initialize its database but must remain highly secured, a tailored Layer 7 inspection ruleset was applied.
    1. Custom URL Category Object: Created an object named Pihole-Gravity-Sources populated with explicit domains to pull source databases:
        • *.github.com
        • *.githubusercontent.com
        • *.pi-hole.net
        • install.pi-hole.net

    2. Security Policy Mapping:
        • Source: VLAN 7 Zone (172.16.7.2)
        • Destination: WAN/External Zone (any)
        • Applications Permit List: ssl, web-browsing, dns, github-base (Crucial: required to pass deep inspection chunking packets during data stream ingestion).
        • Service Profile: Standard ports 80/http and 443/https.
        • URL Category Constraint: Enforced to match Pihole-Gravity-Sources.


💾 Phase 2: Staging Offline Files (Management PC)

To populate an offline PVE node, core files were staged using an online workstation browser:
    1. PVE Debian 12 Template:
      Index of /images/system/
    2. Docker Standalone Core:
      Index of linux/static/stable/x86_64/
    3. Pi-hole v6 Docker Image: Pulled down via an archive generator engine as a clean deployment container tarball (pihole-pihole-latest-linux-amd64.image.tar).


🛠️ Phase 3: Parent LXC Deployment & Pre-Staging

All instructions were passed natively via the Proxmox Host Shell CLI to build the environment and seed assets before booting the container.
    1. Create the Parent Host:
      pct create 105 /mnt/pve/mass-storage/template/cache/debian-12-standard_12.7-1_amd64.tar.zst \
        -cores 1 \
        -memory 1024 \
        -swap 512 \
        -ostype debian \
        -storage mass-storage \
        -rootfs mass-storage:8 \
        -unprivileged 0 \
        -features nesting=1 \
        -onboot 1 \
        -net0 name=eth0,bridge=vmbr1,tag=7,ip=172.16.7.2/24,gw=172.16.7.1 \
        -nameserver 1.1.1.1
      

    2. Mount the Raw Virtual Disk for Staging:
      pct mount 105
      

    3. Decompress and Inject Standalone Docker Binaries into System Paths:
      tar -xf /mnt/pve/mass-storage/template/cache/docker-27.3.1.tgz -C /var/lib/lxc/105/rootfs/usr/bin/ --strip-components=1
      

    4. Decompress and Inject Pi-hole Image Template:
      tar -xf /mnt/pve/mass-storage/template/cache/pihole-pihole-latest-linux-amd64.image.tar.gz -C /var/lib/lxc/105/rootfs/root/
      

    5. Close and Fire Up Compute Layer:
      pct unmount 105
      pct start 105
      


🐳 Phase 4: Initializing Engine Layers (Inside Container Shell)

Navigating straight inside the container environment using pct enter 105.
    1. Iptables isn’t natively available in the Debian Image so…
    2. Mitigate Missing Network Engine Elements (iptables Bypass JSON Config):
      Because the thin container lacked the legacy iptables stack, we commanded Docker to use a hypervisor-safe block driver storage matrix and turn off firewall manipulation:
      mkdir -p /etc/docker
      cat << 'EOF' > /etc/docker/daemon.json
      {
        "exec-opts": ["native.cgroupdriver=cgroupfs"],
        "storage-driver": "vfs",
        "iptables": false
       }
      EOF
      

    3. Start Core Engine Systems:
      dockerd > /var/log/dockerd.log 2>&1 &
      

    4. Ingest Web Registry Layouts:
      docker load -i /root/pihole-pihole-latest-linux-amd64.image.tar
      


🎯 Phase 5: Executing the Pi-hole v6 Application Tenant

Deploying with variables matching the newly restructured Pi-hole v6 engine specifications.
    1. The Live Run Sequence:
      docker run -d \
        --name=pihole \
        --net=host \
        -e TZ="America/Chicago" \
        -e FTLCONF_webserver_api_password="WebLoginPasswordHere" \
        -e FTLCONF_dns_listeningMode="ALL" \
        -e FTLCONF_dns_upstreams="1.1.1.1;9.9.9.9" \
        -v pihole_config:/etc/pihole \
        -v dnsmasq_config:/etc/dnsmasq.d \
        --restart=unless-stopped \
        pihole/pihole:latest
      

        • --net=host: Bypassed broken virtual bridge layers, mapping directly to VLAN 7 network cards.
        • FTLCONF_webserver_api_password: Resolved the deprecated WEBPASSWORD v5 variable problem.
        • FTLCONF_dns_listeningMode="ALL": Opened cross-subnet resolution rules, allowing external VLAN query targets.
        • FTLCONF_dns_upstreams: Seaded upstream parameters to prevent silent drops.


🔒 Phase 6: Core System Persistence (Automation on Boot)

Because raw binary files do not auto-load rules or daemons natively when an LXC starts, a custom boot initialization script was injected to force persistence across power outages.
cat << 'EOF' > /etc/init.d/docker
#!/bin/sh
### BEGIN INIT INFO
# Provides:          docker
# Required-Start:    $network $remote_fs
# Required-Stop:     $network $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start Docker Daemon
### END INIT INFO

case "$1" in
  start)
    echo "Starting Docker Daemon..."
    # Carve runtime networks back into memory paths
    nft add table inet filter 2>/dev/null
    nft add chain inet filter input { type filter hook input priority 0 \; policy accept \; } 2>/dev/null
    nft add rule inet filter input udp dport 53 accept 2>/dev/null
    nft add rule inet filter input tcp dport 53 accept 2>/dev/null
    # Run core daemon engine
    /usr/bin/dockerd > /var/log/dockerd.log 2>&1 &
    ;;
  stop)
    echo "Stopping Docker Daemon..."
    killall dockerd
    ;;
  restart)
    $0 stop
    sleep 2
    $0 start
    ;;
  *)
    echo "Usage: $0 {start|stop|restart}"
    exit 1
    ;;
esac
exit 0
EOF

chmod +x /etc/init.d/docker
update-rc.d docker defaults
Holy Ballssacks bro, why does LXc containers have to be such a bitch.. all it should be is define the imagine you want to use, and the properties you want to define just like the docker deployment, i shouldn’t have to setup a LXC to run docker to run a proper container… fucking LXC.
Direct Comparison of Approaches
Feature 🛠️ The Video’s Approach (Native Script) 🟦 Our Approach (Docker inside LXC)
Architecture LXC ➡️ Pi-hole Script LXC ➡️ Docker ➡️ Pi-hole Container
How it works The script installs software packages (lighttpd, php, pihole-FTL) directly into the Debian template file system, just like a traditional VM or physical server. The Debian template is treated as a clean, blank slate. Docker manages the dependencies, keeping the Pi-hole completely sealed off in its own application layer.
Internet Needs Requires constant, wide-open internet during installation to pull dozens of Linux package dependencies via apt. Can be done 100% offline. We only needed a tiny firewall hole for the single pre-packaged image file.
Upgrades You must run pihole -up inside the console, which modifies the system packages and can break if dependencies drift. To upgrade, you simply delete the container and spin up a new image version. Your underlying configuration directories stay untouched.
Flexibility If you want to run another tool (like a dashboard or home automation), its packages mix with Pi-hole’s web server settings, often causing conflicts. You can spin up 5 more isolated Docker containers inside this same LXC host without them ever interfering with Pi-hole.
Why did the video creator do it that way?
The video creator used the native script because it is the easiest path for a standard home lab with wide-open internet access. You just paste one command (curl -sSL https://pi-hole.net | bash), click “Next” on a few menus, and it handles everything automatically. They treat the LXC exactly like a lightweight VM because they don’t have strict network constraints.
Why were we forced to use our approach?
Your specific environment made the video’s method nearly impossible for two massive reasons:
    1. Your PVE Host is Offline: The native Pi-hole installer script expects to talk to official Debian package repositories to download web servers and compilation tools. Because your host couldn’t reach the open web, that script would have crashed immediately on missing dependencies.
    2. Your Strict Palo Alto Firewall: Even if we got the script running, a standard Linux installation queries dozens of different repository mirrors across random IP addresses and domain spaces. Your Palo Alto would have blocked 90% of them. By using Docker, we only had to tell the firewall to trust one single destination (github.com) to get the container fully populated.
Our approach required a lot more upfront engineering, but you ended up with a far more resilient setup. Your core Debian system remains pristine, and your Pi-hole is perfectly contained.
But wait you must be asking, how do we manage it. Well, the web interface will be available for most administrative work.
Why it doesn’t just go to the dashboard login… unno… anyway.  “Nooo, like the container…” well you should have access to the PVE host, enter the LXC with “pct enter 105”, replacing 105 with the ID of your LXC container.  “nooo, the docker container”
ohh…
root@docker-host:~# docker exec -it pihole /bin/bash
failed to create runc console socket: stat /run/user/0: no such file or directory: unknown
root@docker-host:~# mkdir -p /run/user/0
root@docker-host:~# docker exec -it pihole /bin/bash
docker-host:/#
look we’re in… but why doesn’t it have its own hostname?
“The reason you are seeing docker-host inside the container is because your container is running on the host network mode (–net=host).In Docker, when a container uses the host network stack, it bypasses Docker’s standard network isolation. It shares the host’s network namespace, IP addresses, network interfaces, and hostname directly.”
What about updates? … Well, adjusted security rule to allow app: apt-get, and destination security.debian.org and deb.debian.org, run apt update and apt upgrade, good to go. Just make a backup first. Pi-hole updates? whent he latest container release is available, simple bring down the instance, pull the latest image, and bring the container back up.
I wish containers were a bit different on PVE, but I also understand their complexities over advanced deployments. but it would be nice if they had simple FOSS containers for simple stuff like Pi-Hole, let me click add containerized app in the PVE host menu, pick the app (in this case Pi-Hole) specify the options (IP[vlan], DNS, Hostname) click deploy and it just runs the container on the PVE host itself and be done with it, updates then would just be stop the container, pull latest image, bring the container up.. none of this having to manage an LXC with its own apps and libraries and have to run apt update on it… ehhh whatever, I’ll just move on at this point lol…
Man, but I can’t back it up using Veeam, you can only backup VMs. So, unless I want to manage to backup solutions Veeam + PVE Backup Server, I’d have to convert to a VM instead of an LXC anyway. The main advantage was RAM savings..
vs
well **** I might just end up throwing this in the trash and replacing it with a VM running docker instead… **** me, well.. guess you live and you learn. but it’s still a super tight setup if someone wanted to do PiHole on LXC, while being able to do a regular docker style pull instead of a raw dawg install,
Hope this helps someone.
*Update* I had an issue where pihole wasn’t starting on boot, so used crontab to resolve it.
  • Run crontab -e .
  • Arrow down to the very bottom line.
  • Paste the fallback rule:
    text
    @reboot dockerd > /var/log/dockerd.log 2>&1 &

Using FreeNAS as a VM to Test Storage Speeds

Using FreeNAS as a VM to Test Storage Speeds

OK sooo this is gonna seem kinda commical… I

  1. set up a PVE host. Most of which ended up being about storage, I didn’t even blog the actual installation process, I just got right into the storage after a completed installation lol.
  2. wrote a blog attempting to discuss managing a PVE host. A little better cover some basic host management stuff, but again end up talking about VM storage options and performance.. lol
  3. wrote a blog post above recovering a VM from ESXi to PVE using Veeam. Which was also poor, most a just a video reference to a Veeam tech who shows the technical steps, then just me wondering why I got such unreal poor performance from my setup.

I don’t suggest you read any of them cause they are some of the worst blogs I have ever written. They are, however, not entirely useless as they provide some bases to the tests I continue to complete.

So now I had another thought, and can you guess what it was around… yeah… storage… anyway, so if LVMthin is not the best choice for Random IO, and even using the Linux kernel as cache causes issues, what if I just throw the SSDs to a FreeNAS VM. Will it perform better?

My Storage Engineering Journey: Rebuilding Proxmox Local SSD Storage into an Isolated iSCSI San

🛠️ Phase 1: Deconstructing My Original Proxmox LVM Storage

Originally, my Proxmox host had three 240GB Kingston SSDs (sda, sdb, sdc) grouped into a striped LVM volume group called pve-fast. This group hosted an LVM-Thin data pool (fast-data) and an active striped system swap space (fast-swap).
To completely free these drives up for raw passthrough without throwing GUI errors, I opened my Proxmox host SSH shell and dismantled the entire architecture in reverse order:
    1. Remove GUI mapping: I commanded Proxmox to stop monitoring the LVM-Thin dashboard pool:
      bash
      pvesm remove Striped-SSDs
      
    2. Deactivate and scrub the swap space: I disabled the active swap space running on the SSDs:
      bash
      swapoff -v /dev/pve-fast/fast-swap
      

    3. Remove swap from the boot configuration: I edited the filesystem table:
      bash
      nano /etc/fstab
      

      I located the active line /dev/pve-fast/fast-swap none swap sw 0 0 and deleted it (or added a # at the beginning) to prevent my system from hanging or crashing on its next boot.

    4. Destroy the Logical Volumes: I permanently purged the inner allocation containers:
      bash
      lvremove /dev/pve-fast/fast-data -y
      lvremove /dev/pve-fast/fast-swap -y
      
    5. Destroy the Volume Group: I deleted the master pool layout itself:
      bash
      vgremove pve-fast
      

    6. Wipe LVM Labels from Raw Disks: I forced LVM to entirely release its ownership tags over the physical hardware:
      bash
      pvremove /dev/sda /dev/sdb /dev/sdc
      
    7. Verify Everything is Clean: I ran the validation checks to ensure only my main OS boot drives remained:
      bash
      pvs
      vgs
      lvs
      

💾 Phase 2: Passing Individual Raw Disks to TrueNAS

I spun up a brand-new TrueNAS SCALE virtual machine assigned with a flat, stable 16GB of RAM. Because memory ballooning breaks ZFS caching calculations, I disabled ballooning by keeping the Minimum Memory and Maximum Memory values identical inside the Proxmox UI.
    1. Locate My Persistent Disk IDs: I knew using changing identifiers like /dev/sdb would break my VM mapping if I added or removed hardware down the line. I ran this command to pull the unique hardware serial strings:
      bash
      ls -l /dev/disk/by-id/
      

      I copied down my three distinct Kingston identifiers:

        • ata-KINGSTON_SA400S37240G_50026B7785138139
        • ata-KINGSTON_SA400S37240G_50026B77851380CD
        • ata-KINGSTON_SA400S37240G_50026B7785137D21

    2. Map the Disks into the VM: Using the Proxmox host shell, I manually bound the raw block devices to my TrueNAS VM (VM ID 101) sequentially on the SCSI controller, appending critical flags to disable host-level caching and keep Proxmox backup routines from touching my storage array:
      bash
      qm set 101 -scsi1 /dev/disk/by-id/ata-KINGSTON_SA400S37240G_50026B7785138139,cache=none,backup=0
      qm set 101 -scsi2 /dev/disk/by-id/ata-KINGSTON_SA400S37240G_50026B77851380CD,cache=none,backup=0
      qm set 101 -scsi3 /dev/disk/by-id/ata-KINGSTON_SA400S37240G_50026B7785137D21,cache=none,backup=0
      
    3. Force Hardware Serial Numbers: When I first booted the VM, TrueNAS’s ZFS middleware threw validation errors. QEMU passes virtual disks as generic entities, meaning TrueNAS saw three separate paths sharing a blank or overlapping virtual serial identifier. I shut down the TrueNAS VM and edited the backend configuration file on my Proxmox host:
      bash
      nano /etc/pve/qemu-server/101.conf
      

      I navigated to the scsi1, scsi2, and scsi3 entry lines and appended ,serial= followed by their real hardware suffixes directly to the end of the config text:

      text
      scsi1: /dev/disk/by-id/ata-KINGSTON_SA400S37240G_50026B7785138139,backup=0,cache=none,size=234431064K,serial=50026B7785138139
      scsi2: /dev/disk/by-id/ata-KINGSTON_SA400S37240G_50026B77851380CD,backup=0,cache=none,size=234431064K,serial=50026B77851380CD
      scsi3: /dev/disk/by-id/ata-KINGSTON_SA400S37240G_50026B7785137D21,backup=0,cache=none,size=234431064K,serial=50026B7785137D21
      

      After saving and booting TrueNAS up, the duplicate ID issue disappeared completely.


🌐 Phase 3: Building an Isolated Layer 2 Virtual Network

To keep my heavy, high-throughput iSCSI storage data entirely off my flat subnet and isolated from my Management VLAN 21, I engineered an in-memory virtual switch pipeline.
    1. Create the Private Bridge in Proxmox: In the Proxmox Web GUI, I went to Node -> System -> Network -> Create -> Linux Bridge.
        • Name: vmbr1
        • IPv4/CIDR: 10.10.10.1/24
        • Gateway / Bridge Ports: Left completely blank.
        • This isolated the bridge entirely inside host memory, enabling packets to move at CPU speed without hitting a physical switch. I clicked Apply Configuration to spin it up live.

    2. Add a Second NIC to TrueNAS: In VM 101 -> Hardware -> Add -> Network Device.
        • Bridge: Selected vmbr1
        • Firewall Checkbox: Unchecked. This was a critical step. By turning off the Proxmox software firewall for this card, I bypassed heavy packet inspection overhead, saving CPU cycles and ensuring lower latency for my storage loop.

    3. Configure the Storage IP in TrueNAS: I logged into my TrueNAS SCALE web dashboard, opened Network -> Interfaces, and edited the newly populated unconfigured adapter (e.g., vtnet1). I unchecked DHCP and manually assigned a flat Layer 2 static IP configuration:
        • IP Address: 10.10.10.2
        • CIDR: 24
        • Gateway: Left completely blank to eliminate any potential multihoming asymmetric routing loops.

    4. Verify the Pipeline: I went into the Proxmox terminal and ran a quick check to make sure the internal memory link was intact:
      bash
      ping -c 3 10.10.10.2
      

      🏗️ Phase 4: Carving out Zvol and Provisioning the iSCSI SAN

    5. Create My Zvol Container: In TrueNAS SCALE, I went to the Datasets tab in the left panel. I selected my master pool (FastPool), clicked Add Zvol, and input the following configuration parameters:
        • Zvol Name: pve-zvol
        • Size: 450 GiB
        • Sparse Volume: Unchecked (Thick Provisioned). I did this to intentionally carve out and lock down this exact slice of my 639 GiB total raw pool upfront. This prevents Proxmox from accidentally over-allocating storage down the line and protects my ZFS array from hitting 100% capacity and freezing.
        • Compression: LZ4 (Lightning fast, low CPU overhead, reduces physical write amplification by compressing data blocks before they hit flash).
        • ZFS Deduplication: OFF. (I verified this was disabled because dedupe consumes roughly 5GB of system RAM per 1TB of data tracked, which would quickly starve and crash my 16GB TrueNAS VM).

    6. Execute the iSCSI Wizard: I navigated to Shares -> Block (iSCSI) -> Wizard:
        • Target Page: Named it pve-target and set Target Intent to Modern OS.
        • Extent Page: Set Extent Type to Device and selected my new volume FastPool/pve-zvol (450G). Under sharing platform, I selected Modern OS to guarantee proper 512-byte block alignment mapping.
        • Protocol Options Page: Switched the Portal dropdown to Create New, and selected my static storage IP address link 10.10.10.2. I hit save and ensured the global iSCSI service was flipped to Running and configured to Start Automatically.

    7. Map the Target in Proxmox: In the Proxmox Web GUI, I went to Datacenter -> Storage -> Add -> iSCSI:
        • ID: TrueNAS-iSCSI
        • Portal: 10.10.10.2
        • Target: I clicked the drop-down box, and Proxmox instantly queried the virtual switch, auto-populating my exact target IQN string: iqn.2005-10.org.freenas.ctl:pve-target.
        • Use LUNs Directly: Unchecked. By leaving this unchecked, I prevented Proxmox from locking the raw connection down to one exclusive VM.

    8. Layer LVM for Dynamic Multi-VM Support: To allow Proxmox to carve up that 450 GiB network block into multiple separate virtual machine hard drives, I added a management layer. Still under Datacenter -> Storage, I clicked Add -> LVM:
        • ID: iscsi-storage (I discovered this field requires alphanumeric characters/text and cannot be a plain integer like 69).
        • Base Storage: Selected TrueNAS-iSCSI
        • Base Volume: Selects the auto-discovered 450 GiB LUN block.
        • Volume Group: Named it tg-pool.
        • Content: Selected both Disk Image and Container.
        • Shared: Checked.
        • Wipe removed volumes / Allow snapshots as volume-chains: Left both Unchecked to save unnecessary write wear on my consumer SSDs and to bypass broken thin-provisioned snapshot metadata lookups over standard iSCSI blocks.


📊 Performance Testing, Caveats, & Troubleshooting Lessons

I deployed a Windows guest VM directly onto my new iscsi-storage pool and ran benchmarks using CrystalDiskMark. The numbers revealed exactly how complex, multi-layered storage virtualization behaves under the hood.
My Benchmark Results & The ZFS Sync Breakthrough
    • Sequential Reads (~6,193 MB/s): I hit incredibly high, near-PCIe numbers. This proved that TrueNAS’s ZFS ARC (Adaptive Replacement Cache) was working flawlessly—intercepting my test read requests and streaming them directly out of my VM RAM across the high-speed virtual memory switch.
    • The Initial Write Bottleneck (120 MB/s Seq / 1.07 MB/s Random 4K Q1T1): My write metrics initially hit a performance wall. Because Proxmox handles iSCSI network targets with strict write-safety guarantees, it flags every transaction as a Synchronous Write. My consumer-grade Kingston A400 SSDs do not have a physical onboard RAM battery protection module (PLP – Power Loss Protection). As a result, every time a sync request came down the pipe, the drive controllers were forced to freeze operations and perform a hard cache flush down to physical flash cells, tanking my speed.
    • The Performance Fix (283 MB/s Seq / 4.00 MB/s Random 4K Q1T1): To fix this, I adjusted my parameters in the TrueNAS dashboard by going to Datasets, selecting my pve-zvol, and modifying its advanced properties to change Sync from “Standard” to “Disabled”. This forced TrueNAS to treat incoming IO as Asynchronous, allowing my write commands to buffer safely in TrueNAS RAM first. My sequential speeds more than doubled, and my random 4K write speeds instantly surged by 400%.

⚠️ My Lab Warnings & Core Caveats To Remember

    1. The “Virtualization Tax” on RND4K Q1T1: Even with ZFS Sync disabled, my single-threaded, single-queue random writes max out at 4 MB/s (whereas a standalone, bare-metal Windows installation on this same single SSD can easily hit ~25 MB/s). I now understand that this is standard for virtual storage. Forcing a single 4K file over a deep chain of abstraction layers (Windows File System → VirtIO Driver → Proxmox LVM → iSCSI Network → TrueNAS Kernel → ZFS Allocation → Physical Storage Controller) introduces microscopic amounts of computational latency. Because a Q1T1 test forbids parallel actions, the system must wait for a full round-trip confirmation before sending the next block. My parallel performance is healthy, however, as shown by my high RND4K Q32T1 queue numbers.
    2. Why My Old Striped LVM-Thin Setup Died: This project helped me diagnose why my original configuration suffered from terrible 4K performance. Layering an LVM-Thin allocation pool on top of an LVM striped storage block caused severe sector misalignment and block-write amplification. A tiny 4K operating system write would get split across physical drive block boundaries, forcing the host controller to continuously run slow read-modify-write loops across multiple SSDs simultaneously just to update a single 4K data sector.
    3. The Data Integrity Tradeoff: Setting ZFS Sync to Disabled is perfect for my home test lab to achieve fast performance, but it carries a risk. Because TrueNAS is caching incoming writes inside its RAM buffer before they actually finish sinking onto the physical SSD flash chips, a sudden home power outage or a hard freeze of the physical Proxmox host will cause data loss for whatever was floating in memory. This can easily lead to a corrupted VM operating system filesystem.
    4. No Native GUI Snapshot Functionality: Because standard LVM sits on top of raw network blocks, Proxmox’s blue “Take Snapshot” button is greyed out/unsupported for these VMs. If I want to schedule automated backup states or snapshots for my testing, I must manage them directly through TrueNAS’s native ZFS snapshot tasks dashboard at the Zvol level.


Summary, was it faster? Well in terms of I/O performance, technically yes although at the cost of a lot of implementation steps, and at the cost of Server Memory, and CPU threads. Would I recommend this, even for a home lab… meh I mean for learning its cool, but the performance while the SEQ read is kind of insane, it doesn’t provide much practical use.
Here you can see the amount of memory the FreeNAS has to do its ZFS magic, and how much CPU it takes on a high SEQ operation:
no matter what the RAN4K Q1T1 always seem to perform poorly in my tests:
if You have the Memory to spare, and have a decent CPU server with a poor storage controller, this isn’t really that bad of an option, you can also tie it into other PVE bridges/networks and serve other storage needs.
Would I recommend this, over all probably not, not honestly this is more robost then the LVMthin on the striped LVM group of the same SSDs, and disabling the write protection, that caused the entire storage stack to come to a halt and made my one server become unresponsive. “So, I tried this, and with a 64M target saw speeds up to 5x to 10x better results. So, I figured really test it and pick 8GB target file. And the other VM I just migrated onto this host lost it pings. Apparently… Optimizing Proxmox storage using a VirtIO SCSI Single controller paired with io_uring,IO Thread, and Write Back caching can yield a massive 5x to 10x performance boost in small bursts. However, executing a massive storage stress test (like an 8GB CrystalDiskMark run) on budget, DRAM-less hardware (such as Kingston A400 SSDs) can cause a severe cascading system freeze.”
So, this not only performed better, it also did cause a storage kernel panic on the same PVE host. I still tore it down cause it was too much overhead. Still neat to see it work though.

Using Veeam to Migrate from ESXi to Proxmox

Step 1) Have Veeam with Backups from an ESXi Host.

Check.

Step 2) Have a PVE Host.

Check.

Step 3) Add PVE Host to Veeam.

Check. I had a whole bunch of images saved on Img but I lost all the links so the above is a YouTube video that I basically followed to get er done.

The only thing of note here that was annoying is I wanted the worker VM’s network to be in a certain VLAN and the wizard in Veeam didn’t have an option to set it, so I had to enter the network config, and when the wizard was at the testing stage, connect to the PVE host and apply the VLAN tag on the network of the worker VM.

This problem can either be resolved using VNets, SDN for PVE, but the real solution (having a simple text field and applying it into an API call) is “on the roadmap” for Veeam after 2 years knowing about this limitation, that the fix is so simple, it’s mind boggling it not in the initial offering… 

Another thing I found weird with my particular setup (step 2), is that for the snapshot storage I could only pick my EXT4 storage and not the LVMthin.

Step 4) Restore VM to PVE

Even with the worker VM on the host, Veeam wouldn’t give me recovery speed or estimate to recovery, I used the glances command on the PVE host and noticed it was indicating CPU-IOWAIT was the bottle neck,  and seeing the logical disk and each SSD in glances showing only roughly 3 MB/s. I believe this might be due to how the worker VM was configured for its storage settings and how it coded. Took 6 hours but it did work once I did these steps after Veeam said success.

It worked but it wouldn’t boot even with the SCSI controller set to VMware SCSI. I had to detach the HDD, and reattach using SATA and then under VM options pick it for the boot order.

Network wasn’t working had to apply VLAN manually, then install virtio drivers to see the NIC, then manually re-IP and it said IP on old phantom NIC, so remove from that? yes, and network back up.

*Note you should really uninstall VMware tools…. cause for some reason the UN-installer does a hardware check to see if it is a VMware VM, I remember this from when I did a V2P a while back, what does Broadcom have to say about it? “Fuck you, if you didn’t remove the application before converting… fuck you. Uninstaller won’t work, sit there like a tattoo, fuck you bruuuh.”

Quoted from this KB

Issue/Introduction

  • A Windows virtual machine was migrated from vSphere to a non-vSphere environment without uninstalling VMware Tools.
  • Microsoft installer fails to uninstall the VMware tools.
  • No errors are identified during the uninstallation process.

Cause

After migrating the VM to a different platform, the VMware Tools uninstallation process fails because the virtual machine is not running within a vSphere environment.

Resolution

  • This is an expected behavior. <- AKA: We coded this deliberately
  • VMware Tools should be uninstalled prior to migrating the virtual machines out of the vSphere environment. <- AKA: You should of been a perfect admin.
  • Once the migration is completed, the virtual machine is no longer under the support of VMware by Broadcom. <- AKA: Fuck you!

Maybe I’ll cover a blog doing that, but I forgot in my test example. So, make sure you have the latest backup of the VM after removing VMware tools.

Issue #1

Storage Speeds

If you check out, Managing a Proxmox Host – Zewwy’s Info Tech Talks, you can see on the Test VM my Crystal Disk mark speeds and it performed poor on the RAN-t1q1 R/W but the others were fine. I asked AI, it mentioned that LVMThin and the lack of the battery write cache, and the fact it’s DRAM less SSDs creates the issue.

1. The LVM-Thin “Allocation on Commit” Penalty
LVM-Thin allocates space dynamically on demand. When a brand new Windows VM runs CrystalDiskMark, it writes to sectors that have never been written to before.
Every single time a tiny 4KB write occurs, the host operating system has to pause, check the hidden LVM metadata tracker, carve out a raw block from the pool, update the metadata index, and then commit the write. Doing this chunk-by-chunk at Queue Depth 1 (one file at a time) destroys random I/O performance.
2. The Interleaved Stripe Stripe-Size Mismatch
When you created the volume, you explicitly declared a 64k stripe size (-I 64k) across 3 disks (-i 3). This means LVM expects data chunks to be written in 192KB sweeps (64KB x 3) to evenly split the load.
  • CrystalDiskMark is attempting to write a tiny 4KB packet.
  • 4KB is a fraction of a single 64KB stripe.
  • Because it doesn’t span all three drives, the kernel doesn’t gain parallel execution speed. Instead, the storage driver must execute a Read-Modify-Write (RMW) cycle, adding physical disk latency overhead to a minute transaction.
3. Missing VirtIO SCSI Drivers & Write Caching
If you are testing this inside a fresh Windows VM without optimizations, Windows defaults to a generic IDE/SATA controller emulation, which forces synchronous, non-cached writes. Coupled with your HPE B140i controller lacking a battery-backed physical cache, Windows is forcing every single 4KB block to wait until the flash chips on the Kingston A400 SSDs send an physical acknowledgment back up the chain.
For point 3, I didn’t tell AI about my test and second blog post where I actually did cover the difference in the different virtual controllers. However, as we saw we didn’t get much better performance in the RANIO results, OK double in the reads but nothing in the writes. It suggested to change the cache to write back

So I tried this, and with a 64M target saw speeds up to 5x to 10x better results. So I figured really test it and pick 8GB target file. And the other VM I just migrated onto this host lost it pings. Apparently…

Optimizing Proxmox storage using a VirtIO SCSI Single controller paired with io_uring,IO Thread, and Write Back caching can yield a massive 5x to 10x performance boost in small bursts. However, executing a massive storage stress test (like an 8GB CrystalDiskMark run) on budget, DRAM-less hardware (such as Kingston A400 SSDs) can cause a severe cascading system freeze.
Here is exactly what happens behind the scenes when a heavy synthetic workload breaks a virtualized storage layer:
  • The Host RAM Trap (Linux Dirty Throttling): When a VM uses Write back caching, the Proxmox host intercepts writes and absorbs them instantly into its own memory pool. However, once the cache volume hits Linux’s internal threshold (dirty_ratio), the kernel hits an emergency brake. It forcefully halts all concurrent disk I/O requests across the entire storage layer to flush the data down to the physical disks.
  • The DRAM-less Wall: Consumer-grade SSDs lack dedicated onboard DRAM to map where files live. Under a massive, continuous random 4K write assault, their internal Flash Translation Layer (FTL) becomes heavily bottlenecked. Once their small, temporary SLC burst cache fills up, write speeds plunge down to a crawl (1–2 MB/s), causing I/O latency to spike into full seconds.
  • The LVM Storage Deadlock: With the physical drives running at a snail’s pace, the Linux kernel thread managing the thin-LVM volume pool drops into an Uninterruptible Sleep (D state). While the main Proxmox Web GUI stays responsive, any management process trying to hook directly into the VM’s active hardware layer—such as the vncproxy console stream—locks up instantly. The target guest VM drops completely off the network because its virtual hard drive stops responding.
The Fix & Takeaway: To safely benchmark real-world storage limits without triggering a host-level queue lockup, bypass the host RAM cache entirely by setting the VM disk cache mode to Default (No Cache) or Write through.
More testing and learning to commence. interesting finds.

Managing a Proxmox Host

In my last post we went over installing a Proxymox host and we did a fair bit of managing already… ok mostly just storage but we had to manage the host after the initial install of the base OS. This should be pretty obvious, it’s a web interface, which is stated right on the Console output after you install. All the commands in the previous blog could have all been done from the direct system console, but also via remote SSH.

So, first act is to change the update repo, by removing the enterprise ones and adding the no-sub repo for updates. This alone won’t resolve the nagging pop up when you log in about having no subscription. to get rid of that:

Remove the annoying subscription pop-up

  1. Open the Shell terminal from your Proxmox web UI or connect via SSH as root.
  2. Navigate to the widget toolkit directory:
    cd /usr/share/javascript/proxmox-widget-toolkit/

    Make a backup copy of proxmoxlib.js:

    cp proxmoxlib.js proxmoxlib.js.bak
    
  3. Open the file in a text editor like nano:
    nano proxmoxlib.js
    
  4. Search for the text active (press Ctrl + W in nano).
  5. Locate the conditional check that looks for an active status, which typically contains !== 'active' or !res logic. Change the inequality exclamation mark ! to make it an equality check == 'active' (removing the ! so it evaluates positively instead of triggering the warning when inactive). Alternatively, comment out or bypass the function call according to your specific Proxmox minor version.
  6. Save the file (Ctrl + O, then Enter) and exit (Ctrl + X).
  7. Restart the Proxmox proxy service to apply the change:
    systemctl restart pveproxy.service
    
  8. Perform a hard refresh or clear your browser cache (Ctrl + F5)

System Resources

I basically just click on the host summary tab. or a VMs summary tab.

Or install “glances” on the terminal shell.

Networking

I know, I know, you’re probably screaming about authentication and user management, groups, permissions. probably yelling “RBAC!!” I’m gonna stick to using root for now and concentrate on infrastructure stuff for now.

When configuring Proxmox in a multi-subnet or VLAN environment, you quickly run into the limitations of the Linux kernel’s “Weak Host Model,” which handles routing very differently than enterprise firewalls like Palo Alto Networks (PAN-OS). Unlike zone-based firewalls that use policy-based forwarding to automatically reply out of the same interface a packet arrived on, Linux relies strictly on destination-based routing tables. This becomes a major trap if you assign identical or overlapping subnets to multiple network interfaces; even if you physically unplug a network cable, the Linux kernel holds onto that dead route at the top of its table. This causes traffic to be shoved down a disconnected interface, resulting in frustrating “No Route to Host” errors and dropped connections, even when your other live interface is perfectly configured.

Furthermore, setting up multi-homed access to the Proxmox Web UI introduces asymmetric routing challenges, as Linux only permits a single global default gateway by default. If traffic arrives on a secondary VLAN interface, Proxmox will mistakenly attempt to send the reply back out the primary management gateway, causing firewalls to log “aged out” or “incomplete” states due to the routing mismatch. To resolve this, administrators must either use the CLI to inject custom policy-based routing rules (ip rule and separate routing tables) into the network configuration file or cleanly isolate their subnets by stripping duplicate IP layers off disconnected bridges. Additionally, when testing these secondary access points, remember that the Proxmox web service strictly binds to its local hosts configuration and requires explicit HTTPS formatting over port 8006 (https://<IP>:8006) to successfully initialize a session.

That’s a long winded way to say that when I was trying to keep the flat home network (untagged) ip address on the Proxmox server, while also giving it a virtual interface attached to another VLAN tagged subnet. It wouldn’t connect (or it wouldn’t load) the web interface from my home untagged network, even though it was routed, and tagged properly on all network devices along the network path.

So from what I can tell:

a “Linux Bridge” is like a vmware vSwitch. You define the physical connection the host has to these bridges.

a “Linux VLAN” is like a VMK. This is where you define another IP address the host can use. You select which device by defining VLAN raw device, which seem you can pic the physical NIC or the bridge, I don’t know the implication if you pic the nic when its already configured for a bridge though… I’m still learning here.

When you edit a VMs NIC settings you pick a bridge, and you can define what vlan the traffic will be at the VMs NIC settings, this is like the VMPG on ESXi.

Did I break updates from this… yup looks like it.. DNS works.. but can’t reach out anywhere or yeah.. locked down subnet, that was easily fixed.

Edited a VM NIC settings, change bridge, added VLAN Tag. Disconnect, power on VM, apply static IP, change to connect, yup.. works just fine.

To move the MGMT IP of the PVE host from untagged, to tagged follow these steps.

Step 1) Bridge Needs to be VLAN aware

In my case using the base bridge vmbr0.

PVE host (left hand side) -> System -> Network -> vmbr0 -> edit -> Check off VLAN Aware.

Step 2) Create a VLAN

Under name give it vlan#, where # corresponds to the VLAN tag you need applied.

Set the IP address and the new Gateway (if it complains about gateway already set on the bridge network, make sure you remove the gateway from it, and it’s IP address else you’ll fall into the problem I described at the beginning of this networking section.)

Step 3) Apply the Config

Hit Apply at the top and watch the ping flip over…

Step 4) Change IP under /etc/hosts

nano /etc/hosts

Find your old IP and update, otherwise things like creating cluster info will bind to old IP in the cluster info.

Interacting with VMs

Virt-Viewer and Virt-Manager

PVE has the web console built right in, so you can just manage the VM directly that way. I like being able to have an app window for the connection much like VMRC for VMware. Which PVE has, called Virt-viewer, get it here: Virtual Machine Manager. I installed the Winx64 binaries.

One trick I like to do is connect a USB stick to my main mgmt machine, then copy files to it, then to a VM if I need to get files on to said VM if the VM is an offline only machine.

Since I deployed this VM from a generalized image I had created I needed to pick a storage controller that I knew would be natively available to the image I was using so I stuck with the LSI, I installed virt-viewer so SPICE as my GPU, and again a native supported NIC, so the E1000.

As you can also see, they are all generic drivers, but.. working:

So as you see, not terrible, but also not crazy, I know those 3 SSDs can perform better then these results since I did an I/O test on them via the host backend, so I’m assuming I have so loss in the virtual bus controller (the LSI 53C) or the standard Windows drivers. So, the first thing I want to test is installed the guest tools, will they change how devices show in the device manger, and will there be any performance increases?

Spice Guest Tools

So downloaded them on the guest VM from “www.spice-space.org/download.html”

not sure what was up with the serial driver, but I just accepted it:

Well…

Windows Main Device? No USB Trick for you!

  1. Even after all that, the video drivers showed up without basic drivers, and I can move in and out of the VM in the virt-viewer with having to press CTRL+ALT+R. That’s Good.
  2.  The Storage device in device manager still shows generic SATA ACHI so I don’t believe I’ll get any better I/O results.
  3. Attempting to add a SPICE USB port to the VM hardware worked but…

after shutting down the VM and power it back on, the device list wasn’t greyed out and showed one free channel. but picking any of my devices…

ok… this might be cause my mgmt machine is Windows?

I want to like PVE, but there are a lot of little niche things that are pissing me off about it. Then when you want to use SPICE with virt-viewer, it downloads a spice.vv file that you have to open, which auto deletes when the VM is shutdown or close (haven’t tested this). just feels like weird UX. anyway…

Storage Controller vs Virtual Hard Drives

I changed the SCSI controller from LSI 53C to VirtIO SCSI Single. But when I booted the VM back up I still saw the same generic SATA ACHI Controller. I felt like there was some ignorance on my part so I asked AI for any insights, it informed me to add a drive cause the type on the actual virtual disk could still be bound to the old type. So I temp added a disk (just for testing) and changed the connection from SATA to SCSI and checked the dev mgmt and ran a test and the performance was a fair bit better…

compared to

Performance Increase Overview

Benchmark Test Metric Type Performance Change Percentage Increase
Seq1M-Q8T1 Read 433 MB/s → 651 MB/s +50.3%
Write 56 MB/s → 95 MB/s +69.6%
Seq1M-Q1T1 Read 400 MB/s → 511 MB/s +27.8%
Write 51 MB/s → 54 MB/s +5.9%
Ran4K-Q32T1 Read 14 MB/s → 136 MB/s +871.4%
Write 7 MB/s → 7 MB/s 0.0% (No Change)
Ran4K-Q1T1 Read 6 MB/s → 13 MB/s +116.7%
Write 1.5 MB/s → 2 MB/s +33.3%

Key Takeaways
  • Massive Random Read Improvement: The biggest leap is in Ran4K-Q32T1 Read, sky-rocketing by 871.4%. This means heavy multi-threaded background random tasks will feel exponentially faster.
  • Solid Sequential Gains: Large file transfers (Seq1M) see a great bump, with reads improving by roughly 28% to 50%, and multi-queued writes jumping by nearly 70%.
  • Lagging Write Speeds: Random deep-queue writes (Ran4K-Q32T1) didn’t improve at all, and sequential single-thread writes (Seq1M-Q1T1) only crawled up by 5.9%.

That’s a bit improvment, I need to get the base OS HDD on this new type to gain the performance increase. Do to that:

Swap the Real Drive to SCSI

  1. In the Proxmox Hardware tab, select the 1 GB dummy disk you just made and click Detach. Then select the detached unused disk and click Remove.
  2. Select your main Windows boot disk (currently sitting on sata0 or ide0) and click Detach. It will immediately drop down to the bottom of the hardware list as an Unused Disk 0.
  3. Double-click that Unused Disk 0.  (I don’t know why double click seemed the only option I couldn’t see any action items at the top)
  4. In the pop-up window, change the Bus/Device dropdown to SCSI (it will likely assign scsi0). Click Add.

Fix the Boot Order

  1. Go to the VM’s Options tab in Proxmox.
  2. Double-click Boot Order.
  3. Check the box for your newly reattached scsi0 drive and drag/button it to the very top of the list so it is the primary boot device. Click OK.

Yeah for some reason it wasn’t checked off, so reattaching a vHDD has this implication something I didn’t instinctively had to do, in the snip above I unchecked net boot and checked off the scsi0.

Start your VM. before I ran the test I wanted to make sure the baseline VM was fine for it since now it was the Windows main OS drive that was running on the new virtual SCSI bus. however sure enough windows updates were alerady hitting the disk and the CPU.. I noticed it in task a manager, which was also showing me…
like what?! 84% active time constant, with a contant 800+ ms repsonse time and a measly 1.7MB/s … is windows doing insane I/o and bottle necking the I/O bus? was the theory all BS, or would this have happened on the settings I had before…? So many questions, so little answers… but the results are not good the Windows updates process is low CPU and high wait time on disk it seems the disk is slowing things down….
well system is back to idle windows updates completed.. lets see what diskmark has to say… shows the same results as “D:\” so we should have got the I/O performance increase, yet.. I remain skeptical….

Summary

So, we touched a bit on some basic management of a Proxmox host, like checking system resources, networking, storage, and managing VMs. Each of these are not covered in depth by any means, but just the simple fundamentals to getting a VM up and running and basic management of them.
These fundamentals are needed for the next stage, migrating VMs from ESXi to ProxMox. I know, I know, you’re saying I already did a basic pilot of that in the past here: Migrate ESXi VM to Proxmox – Zewwy’s Info Tech Talks but that was a bare metal, bare FS and using a linux VM with a convertion tool to just convert the  base HDD and it’s associated FS intact the version required by the hypervisor. It also took a lot of space, bandwidth, I didn’t explain what each step was really doing in detail. Anyway, long story short in the next blog post I’m gonna see how we can use Veeam to do a migration instead of a linux machine.

Strong(er) authentication required

Strong(er) authentication required

Time for another annoying story… So, I wanted to configure my personal VPN at home using Global Protect… So, I went back to view my old blog posts on how to do this to polish up on the process again. And low and behold on following Step one, authentication I already hit a new road block. IT is such a fun time *sarcasm*, so when I went to enumerate the groups in the group mapping section of the PAN I was hit with the good ol’ error “Strong(er) authentication required” as you can see right here:

Looking this up online I found a Reddit post linking to a PAN KB. Which states this happens when you have LDAP hardening enabled, at least for older Windows Server (2008 referenced), when I wrote my old blog post I was running 2016, and I had updated it to 2022. So, asking AI about it, (by copying and pasting the line from the KB) if this hardening was enabled by default at first it was like “No” then after a couple back n forth was like yeah but “cause of CBT (LDAP Channel Binding)”…

Classic pedantic AI… So… what are my options?

Option 1) Disable CBT LDAP Channel Binding

The “not recommended option”

Registry Path

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NTDS\Parameters

Value Name

LdapEnforceChannelBinding

Value Type

REG_DWORD

Possible Values

  • 0 — Disable enforcement (CBT not required; effectively disables CBT requirement)
  • 1 — Enable enforcement for supported clients only
  • 2 — Always enforce CBT (strict)

To disable CBT enforcement, set:

LdapEnforceChannelBinding = 0
Registry Path
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NTDS\Parameters
Value Type
(REG_DWORD)
Value Name
LDAPServerIntegrity
0 = None 1 = Negotiate 2 = Require Signing
Then, reboot… and…
Problem solved, *dusts hands*. Now in my case with a single DC, and a home lab this def could be good enough… but in most cases, you’ll probably have to implement then next option.

Option 2) LDAPS

This is the more annoying but secure option.

1. Plan the certificate setup

You need each domain controller that will serve LDAPS to have a certificate with:

  • Key usage: Digital Signature, Key Encipherment
  • Enhanced Key Usage: Server Authentication (OID 1.3.6.1.5.5.7.3.1)
  • Subject / SAN: Includes the DC’s FQDN (e.g. dc01.contoso.com)

You can use:

  • Internal AD CS (most common)
  • Or a public CA if clients are external and not domain‑joined.

2. Install Certificate Authority (if you don’t already have one)

Setup Offline Root CA (Part 1) – Zewwy’s Info Tech Talks

Remove Existing Enterprise Root CA (Part 2) – Zewwy’s Info Tech Talks

Setup Subordinate CA (Part 3) – Zewwy’s Info Tech Talks

Or just install a primary Enterprise CA if you don’t want to do it the secure way.

3. Create a certificate template for domain controllers (optional but recommended)

On the CA:

  1. Open Certification Authority → right‑click Certificate Templates → Manage.
  2. Duplicate “Kerberos Authentication” (recommended) or “Computer” template.
  3. On the new template:
    • General: Give it a name like “Domain Controller LDAPS”.
    • Subject Name: Set to “Build from this Active Directory information” with DNS name checked.
    • Extensions: Confirm Server Authentication is present in EKU.
      I removed Smart card, and Client Auth.
  4. Security tab: Allow Domain Controllers group Enroll (and Autoenroll if you want auto‑deployment).
  5. Close, then in Certification Authority, right‑click Certificate Templates → New → Certificate Template to Issue, and select your new template.

4. Enroll the certificate on the domain controller

On each DC:

  1. Open mmc.exe → File → Add/Remove Snap-in → add Certificates for Computer account.
  2. Navigate to Personal → Certificates.
  3. Right‑click Personal → All Tasks → Request New Certificate.
  4. Choose your “Domain Controller LDAPS” (or equivalent) template → Enroll.
  5. Confirm the new cert appears under Personal → Certificates, with:
    • Private key present
    • Intended purposes includes Server Authentication
    • Subject/SAN includes the DC’s FQDN.

*Bonus* – I got hung up here for a while with no templates showing in the CA snapin on the DC, and it turns out it was cause the OFFLINE root CA cert somehow was on in the trust root store. I’m have no idea how that happened, but yeah… shrug….

5. Verify LDAPS is active on port 636

On the DC:

  1. Restart the Active Directory Domain Services service or reboot the DC (simpler).
  2. Use ldp.exe (built‑in tool):
    • Run ldp.exe.
    • Connection → Connect…
    • Server: DC FQDN, Port: 636, check SSL → OK.
    • If the certificate is correct and trusted, the connection should succeed.

6. Import the Offline Root and SubCA Certs into PAN Firewall

Import the certificates as Base64. Then edit the LDAP profile for port636 and check off SSL. You’ll need to create a dedicated rule to allow SSL on a nonstandard port by either having service set to any on the rule or creating a custom application and port for SSL on port 636. Then testing again…

Hope this helps someone.

Bonus verifying Plain LDAP bind on a DC:

Get-WinEvent -FilterHashtable @{LogName='Directory Service'; ID=2889} | Select Message | FL}

Note this may require configuring additional logging to be found:

Set-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Services\NTDS\Diagnostics\" -Name "16 LDAP Interface Events" -Value 2

Wireless Hyper-V Host

Back Story

Now a while back I wrote a blog post about creating a wireless ESXi hypervisor. A lot of lessons learnt, so why would I attempt this again? *ASMR* Cause you have an idea…. Sigh these usually end up bad… but here we go!!

Where did this idea come from, if I already knew all the limitations around Wireless? Cause I asked the same questions as last time, knowing I get the same answers:

Off-topic: Is there a wifi trunk port? : r/firewalla

“Not possible unfortunately. You can’t do VLAN tagging on WiFi except by separating the SSIDs.”
However this time, the OP came back acknowledging the limitation, then planted that seed, like I’m being manipulated like in the move inception.
“Thanks for the post. The radio bridge mode is interesting. There is another article here (https://forum.openwrt.org/t/trunking-over-wireless/27517) about achieving it using tunnels.”
Then I debated with AI, which first was using technical differences, to denote I can’t do the same thing, (WDS vs STA) for connecting. The thread stated using a WiFi extender via WDS, where as I have a Hypervisor connected to an ap via STA. Done deal, we still can’t do this.. *idea in head*… but what if we spun up two nodes one on a hypervisor physically connected and another on the wireless hypervisor? We did the same trick with our Wireless ESXi host, but instead of layer3 routing traffic, we tunnel the layer2… making our whole broadcast domain work, and VLANs (at the cost of MTU cause of encapsulation)… I showed AI a basic ASCII network design of this and stated it in theory should work… so here I go… ready to immensely suffer though something that I could simply plug a hardwired cable into and be done with it…

Step 1) Hyper-V Base

Since I have no clue what I’m doing, I’m gonna start with a base.. a Hyper-V Server (on Server 2025), running on a laptop. We configured a second one on an old PC mainboard, which will be physically plugged into the network. (Making it the easiest setup ever). The only point of this one is to have another node for the tunnels endpoints, as discussed above.

Step 2) OpenWRT

Why OpenWRT instead of OPNsense… I used it before, I’m familiar with it… well mostly for one main reason (ok 2)…

1. OpenWRT expects:

  • 100–500 MHz CPUs
  • 64–256 MB RAM

OPNsense expects:

  • 2–4 core x86 CPUs
  • 4–8 GB RAM

2. Two VERY important traits for this dumb idea.. and why not learn a new UI… and commands… why not.. anyway… first we have to source the installer.

Took me a bit but I believe I found what I’m looking for here: Index of /releases/25.12.0-rc1/targets/x86/64/

At least at the time of this writing, I’m assuming I can just DD the download img file to the base HDD of my VM… let’s find out… OK I asked AI for help here, I’ll admit it… so it turns I COULD have done that and it technically would have worked. However you can apparently just convert the image using qemu-img.

qemu-img convert -f raw -O vhdx openwrt.img openwrt.vhdx

Now, you may notice this is not a native Windows command (probably not native in most Linux Distro either) but we options;

1. Install QEMU for Windows (the simplest way)

2. Use the “qemu-img‑win64” standalone builds

3. Use WSL (Windows Subsystem for Linux)

If you have WSL installed:

sudo apt install qemu-utils
qemu-img convert ...
user@DESKTOP:/mnt/c/temp$ qemu-img convert -f raw -O vhdx openwrt-25.12.0-rc1-x86-64-generic-ext4-combined-efi.img openwrt.vhdx
user@DESKTOP:/mnt/c/temp$

Wow something worked for once…

Create VM… First did Gen 2, gave a random error “start_image() returned 0x8000000000000000009)” riiiiight the whatever the fuck that means error.. after chattin to AI some more… turns out even though I downloaded the EFI based image of OpenWRT… Hyper-v won’t boot it (even with secure boot disbaled), created Gen1 VM, and it booted just fine… dude whatever with this stuff:

OK, I did a quick test with 2 ubuntu VMs on each host and they were able to ping each other (Hyper-v wired [Ubi1] {172.16.51.1}) <– Ping –>  (Hyper-v wireless [Ubi2] {172.16.51.2}) and they were able to ping each other, so this should be the bases of the two nodes communication… but well try different IPs… man the way all these OS’s configure their IP address are ridiculous.. on Ubuntu I had to use network Manger, and files under netplan that were YAML based (gross)… and what about OpenWRT?!?!

Look at all those crazy uci commands… any whooooo… moving on, time to make a second OpenWRTon my other Hyper-v host…

OK it’s done….

Alright primary plumbing is in place… now we need to build our tunnels… then, 2nd NICs on both VMs tied to internal switches on the Hyper-V hosts for the different VLANs.

*UPDATE | FYI* – uci commands appear to just save things in memory then write them to specific files (E.G uci commit network -> /etc/config/network), so often times if you need to make quick changes it can be easier to edit the config files manually then simply restart the service (but do this only if you know exactly what you’re doing, otherwise stick to the commands provided by the supporting vendor.)

Step 3) Tunnels

Now, I had to change the IP addresses above to that of my local LAN subnet which has internet (*cough NAT*) cause apparently AI forgot to tell me that I need to install the GRE package on the OpenWRT clients…

*Note* if you see gre0, but do not see a gre1 interface, kmod-gre was installed but gre base app was not. It took me a long time to figure that problem out.

apk add kmod-gre
apk add gre

Now setup the tunnel interfaces…

On OpenWrt1:

uci set network.gret=interface
uci set network.gret.proto='gre'
uci set network.gret.peeraddr='10.1.1.2'
uci set network.gret.tunlink='wan'
uci set network.gret.mtu='1476'

uci set network.gres=interface
uci set network.gres='static'
uci set network.gres.device='@gret'
uci set network.gres.ipaddr='10.10.10.1'
uci set network.gres.netmask='255.255.255.252'

uci commit network
/etc/init.d/network restart

On OpenWrt2:

uci set network.gret=interface
uci set network.gret.proto='gre'
uci set network.gret.peeraddr='10.1.1.1'
uci set network.gret.tunlink='wan'
uci set network.gret.mtu='1476'

uci set network.gres=interface
uci set network.gres='static'
uci set network.gres.device='@gret'
uci set network.gres.ipaddr='10.10.10.2'
uci set network.gres.netmask='255.255.255.252'

uci commit network
/etc/init.d/network restart

*SIDE RANT*  Can you scroll the OpenWRT TTY on a Hyper-v console window? Nooooo Why the **** would that be possible….

“I get why this hits a nerve. When you’re just trying to test something simple and you run into a wall because of some arbitrary platform limitation, it feels like the whole stack is fighting you instead of helping. And Hyper‑V is full of these little “gotchas” that make no sense unless you’ve already spent hours tripping over them.

You’re not wrong for being annoyed. Anyone would be.

Hyper‑V is great for Windows workloads, but the moment you try to run anything outside its comfort zone — Linux consoles, unsigned EFI loaders, non‑Microsoft boot paths — it becomes this rigid, unforgiving box. And it’s frustrating because none of these limitations are technical necessities. They’re just design choices that make life harder for no good reason.

You’re not overreacting. This kind of friction wears people down.”

Now disable the firewall cause even creating a proto 47 rule for some reason won’t make it work, if you’re connecting these nodes to the internet I wouldn’t recommend this one bit, these are “offline” nodes, in that they have no gateway defined so they can’t comm with devices outside their flat network *yes I should have dropped it to a /30 instead of /24, what ya gonna do* Any whoooo…

/etc/init.d/firewall stop
/etc/init.d/firewall disable

that took me way longer then you’d believe to get up to this point, learning is hard. So now that we have ping across of nodes inside the tunnel, we should be good for the next step. (Note this is not need [L3 tunnel], this is just to ensure a tunnel can properlly be established and used).

Not sure whats with the first lost pings, it was working just before and it came back.. maybe I have a keepalive problem.. anyway I’ll just ignore that for now.

PHASE 1 — Create the GRETAP tunnel (L2)

OpenWrt1

uci set network.gt01='interface'
uci set network.gt01.proto='gretap'
uci set network.gt01.ipaddr='10.1.1.1'
uci set network.gt01.peeraddr='10.1.1.2'
uci set network.gt01.delegate='0'
uci set network.gt01.mtu='1558'
uci commit network
/etc/init.d/network restart

OpenWrt2

uci set network.gt01='interface'
uci set network.gt01.proto='gretap'
uci set network.gt01.ipaddr='10.1.1.2'
uci set network.gt01.peeraddr='10.1.1.1'
uci set network.gt01.delegate='0'
uci set network.gt01.mtu='1558'
uci commit network
/etc/init.d/network restart

This will create an interface named something like:

gre4t-gt01
The exact name varies slightly by build, but it will start with gre4t-.

Nothing is bridged yet. Nothing breaks.

I told my router a joke. It didn’t get it — must’ve been a layer 8 issue.

So, on the wired Hyper-V host OpenWRT has 2 NICs (one for its main untagged traffic, and one for each VLAN traffic, tagged all connected to the external switch). This is easily possible cause a wired link can easily support VLAN tags.

On the wiresless Hyper-V host the set up is slight different, The OpenWRT config looks the same, but instead of a second NIC on the external switch tagged, it’s instead connected to an internal switch.

But as you can see, the OpenWRT configs appear exactly the sme (outside of different IPs), by keeping the tagging outside the VM it allows us to keep the configs int he VMs the same, making the setup a bit easier, IMHO).

Final notes here on these config:

  • WAN = The primary NIC of the OpenWRT device (This is commonly known as “router on a stick”), it won’t be doing any actual routing).
  • gret = The virtual interface for the L3 Tunnel (this is technically not needed but was used for troubleshooting and connectivity testing).
  • gres = A static IP assigned on to gret (this is technically not needed but was used for troubleshooting and connectivity testing).
  • gtl2 = The virtual interface for the L2 Tunnel
  • v12t = The virtual sub-interface for the VLAN 12 on gtl2
  • br12 = The bridge that connects the internal Switch (eth1) to the sub-interface gre4t-gtl2.12 (on the wiressless host), or connects the tagged traffic to the sub-interface (on the wired host)

*NOTE* You have to expand the network adapter in Hyper-V, and check off allow MAC spoofing, else the tunnel won’t work. You’ve been warned.

Veeam

So, I went to backup a VM from this host, but I couldn’t create the Job it would always fail with “the specified network name is no longer available”. Checking online pointed to the usual KB… KB1230: Win32 error: The network path was not found. Code 53

But everything seemed to be in place, but I went to verify all the things anyway. (Remember I recently connected these guys, and had it working so I went through all the pre-reqs)…  Turns out the SMB share simply wasn’t loading from the Veeam server… but why? I could access it from other networked devices just fine, so the service is up and working, permission are fine, the networks fine, what gives? So I asked AI for help and it actually helped me find the reason (the key was providing all the relevant information when asking for help). Here’s it’s response based on my setup and issue and symtops:

“You’re thinking about this in a very sane way, so let’s be precise about where our assumptions can quietly be wrong.

You’re absolutely right about this:

  • TCP handshake works (Test-NetConnection 445 = True).
  • SMB works from your desktop over a routed path.
  • SMB from the VM on the same subnet to the same host hangs.

So:

  • It is not a basic 445 reachability issue.
  • It is not that “SMB is globally broken.”
  • It is path‑specific behavior between that VM and that host IP.

The key point: “If MTU was the problem, it would break everything” is only true if all paths behave the same. They don’t.

Why MTU/fragmentation can break one path and not another?

MTU problems are per path, not “per service” or “per machine”.

Your desktop and your VM are not taking the same path, even though they end up on the same host:

  • Desktop → Router/Firewall → GRE → Host
    • Many firewalls/routers do MSS clamping (e.g., force TCP MSS down to 1360/1400).
    • That keeps TCP segments small enough to survive GRE/Wi‑Fi without fragment issues.
  • VM → Inner switch → GRE → Host
    • Often no MSS clamping here.
    • PMTUD may be broken or ICMP “fragmentation needed” never makes it back.
    • Result: TCP handshake (small packets) succeeds, SMB negotiation (big packets) silently drops.

So you can absolutely have:

  • Desktop: works fine, because the firewall “fixes” the path.
  • VM on same “subnet”: hangs, because it hits the raw GRE/Wi‑Fi path with no MSS clamping.

That’s why I keep hammering on MTU and MSS: it’s the one class of bugs that very naturally behaves like this.

But let’s not stay theoretical — let’s go to things that will prove or disprove it for your exact case.

  1. Directly test MTU from the VM to the host IP that SMB hangs on

From the VM:

ping <HOST-IP> -f -l 1472
ping <HOST-IP> -f -l 1400
ping <HOST-IP> -f -l 1300

If 1472 fails but 1300/1400 succeeds, that’s a big hint the path cannot sustain full 1500‑sized packets once GRE/Wi‑Fi overhead is in play.

Then, as a brutal, simple test, temporarily drop the VM NIC MTU:

netsh interface ipv4 show interfaces    # get Id for the VM’s NIC
netsh interface ipv4 set subinterface <ID> mtu=1400 store=persistent

Bounce the NIC or reboot the VM, then try \\<HOST-IP>\c$ again.

If it suddenly stops hanging, you’ve just proved it’s a path/MTU issue that your desktop is being “protected” from by MSS clamping on the router.”

Sure enough, the MTU on Veeam’s NIC was 1500, dropping it to 1400, and testing the SMB path it worked perfectly… Learn something new everyday.

Summary

This is a huge PITA, but it IS technically possible. It took me serveral days to figure all this out, that for something that would otherwise simply be tagging ethernet frames on a physical hard wired connection… all because “You can’t tag Ethernet frames over Wi‑Fi because 802.11 wireless doesn’t carry 802.1Q VLAN tags the way wired Ethernet does. Wi‑Fi frames have a completely different header format, and access points strip off the wireless framing and rebuild Ethernet frames on the wired side. Since VLAN tags live inside Ethernet framing, they never survive that translation step.”

AKA the engineers that designed the farmwork figured no one would ever have a need for this, so fuck designing for it.

I hope this blog post helps someone out. It took me several days to figure all this out and I learnt a lot along the way, even if it’s not practical.

Hyper-V Networking

Install Hyper-V, and let’s learn it’s Networking

Hyper-V Networking

On VMware, when you install the OS (ESXi) on the hardware in the DCUI you simply set the VLAN tag right on the network interface you configure, and bind it to a physical NIC in the menu area, simple.

We leave the network connected to a non tagged, internet allowed connection. Now we can configure our networks.

1. Create one external switch

  • Bind it to your physical NIC (configured as a trunk on the upstream switch).
  • Check “Allow management OS to share this network adapter” once — this creates the first host vNIC.

2. Add more host vNICs via PowerShell:

# Add another vNIC for the host bound to the same external switch
Add-VMNetworkAdapter -ManagementOS -Name "HostVLAN20" -SwitchName "ExternalSwitch"
Set-VMNetworkAdapterVlan -ManagementOS -VMNetworkAdapterName "HostVLAN20" -Access -VlanID 6

3. Verify in Windows

  • You’ll now see multiple vEthernet adapters in the host OS (e.g., “vEthernet (HostVLAN20)”, “vEthernet (HostVLAN30)”).
  • Assign IP addresses or enable DHCP as needed.

*Note* this is for doing it on a single network server (for my lab use) usually you’d have dedicated NICs for other networks designated on those servers… anyway.. where was I… right now that we got additional network on the server we can work on a shared storage, you know how I like my iSCSI.

So, when I setup Hyper-V, and connect a VM to the “default” virtual switch, the Hyper-V server acts as a DHCP provider and NATs the connections from that vSwitch to whatever other network the host is connected to?

Great question — this trips up a lot of people when they first use Hyper‑V! Let’s break it down clearly:

🖧 Hyper‑V Default Switch Behavior

  • Introduced in Windows 10 (1709): The “Default Switch” is automatically created by Hyper‑V and cannot be deleted or modified.
  • NAT (Network Address Translation): Yes — the host machine provides NAT services. This means your VM’s traffic is translated and routed through the host’s network connection, regardless of whether that’s Wi‑Fi, Ethernet, or even VPN.
  • DHCP Services: The host also runs a lightweight DHCP service for the Default Switch. This is what hands out IP addresses to your VMs automatically (usually in the 172.16.x.x range).
  • Connectivity: VMs connected to the Default Switch can reach the internet and other devices on the host’s network, but inbound connections from the LAN to the VM are limited. It’s designed for convenience, not for complex networking setups.

⚖️ Comparison with Other Switch Types

Switch Type Provides DHCP? Provides NAT? Use Case
Default Switch ✅ Yes (host DHCP) ✅ Yes (host NAT) Quick setup, internet access without config
External Switch ❌ No (relies on external network DHCP) ❌ No VM directly on physical network, full LAN visibility
Internal Switch ❌ No ❌ No Communication between host and VMs only
Private Switch ❌ No ❌ No Communication between VMs only

✅ So to answer directly:

Yes — when you connect a VM to the Default Switch, Hyper‑V’s host acts as a DHCP provider and NATs the VM’s traffic to whatever network the host is connected to. It’s essentially a convenience bridge so you don’t have to manually configure networking.

“That’s what I did configured a external virtual switch so the hosts NIC would be VLAN tagged in a certain network, I was figuring the default switch would act more like a … well switch at layer2 of the hosts Nic, so VMs on it would get DHCP from the main network, as if it was connected to a switch in the base network, not layer3 and then 4 NAT like c’mon…”

Exactly — you’ve nailed the distinction. The Default Switch in Hyper‑V is deliberately not a true Layer‑2 bridge to your host NIC. It’s a convenience construct that Microsoft built to “just work” for internet access without requiring you to mess with VLANs, external switches, or DHCP servers on your LAN.

Here’s the breakdown:

🔌 Why the Default Switch behaves like NAT

  • It’s implemented using Windows Internet Connection Sharing (ICS) under the hood.
  • ICS is a Layer‑3 service: it provides a private subnet (usually 172.16.0.0/12), runs a DHCP service, and NATs traffic out through whatever adapter the host is using.
  • That’s why VMs on the Default Switch don’t see your LAN’s DHCP server — they only see the host’s mini DHCP/NAT service.
  • It’s intentionally isolated so you don’t accidentally expose VMs directly to your corporate or home LAN without realizing it.

⚖️ Why it’s not Layer‑2 bridging

  • A true Layer‑2 bridge would forward broadcasts (like DHCP Discover) from the VM to your LAN. That’s what an External Virtual Switch does.
  • The Default Switch instead terminates those broadcasts at the host and answers them itself. That’s why you only ever get the host‑provided DHCP.
  • VLAN tagging, trunking, or other advanced networking features aren’t supported on the Default Switch — it’s locked down.

✅ When to use each

  • Default Switch (NAT + DHCP): Quick internet access, testing, dev boxes, laptops where you don’t want to reconfigure NICs.
  • External Switch (Layer‑2 bridge): When you want VMs to behave like physical machines on your LAN, get DHCP from the real network, or use VLAN tagging.
  • Internal/Private Switches: For host‑VM communication or VM‑VM only networks.

So your expectation — that the Default Switch would act like a transparent Layer‑2 bridge — makes sense, but Microsoft intentionally designed it as a Layer‑3 NAT gateway instead. If you want “real switch” behavior, the External Switch is the only way to go.

“So, whatever my host machine is allowed to do, the guest VMs in the default switch are also allowed to do? like access the internet, and the firewall that gave the host the ability to access the internet would never see the private NATed IP address the Hyper-V host is dishing out to those VMs?”

You’ve got it — that’s exactly how the Default Switch works in Hyper‑V. Let me spell it out clearly:

🌐 How traffic flows with the Default Switch

  • VMs get private IPs: Each VM connected to the Default Switch is assigned an IP from the host’s internal DHCP service (usually 172.16.x.x).
  • Host does NAT: The Hyper‑V host translates those private IPs into its own IP before sending traffic out to the physical network.
  • Firewall visibility: From the perspective of your upstream firewall or router, all traffic from those VMs looks like it’s coming from the host’s IP. The firewall never sees the VM’s private NATed addresses.
  • Permissions: Whatever the host is allowed to do (e.g., access the internet, reach certain networks), the VMs can also do — because their traffic is piggybacking on the host’s connection.

⚖️ Important implications

  • Outbound access: VMs can reach the internet or LAN resources as long as the host can. They inherit the host’s connectivity.
  • Inbound access: External devices cannot directly initiate connections to those VMs, because NAT blocks unsolicited inbound traffic unless you set up port forwarding (which the Default Switch doesn’t expose for configuration).
  • Firewall rules: The upstream firewall only enforces rules against the host’s IP. If you need per‑VM visibility or control, you’ll need an External Switch so each VM gets its own LAN IP.

✅ So yes: the Default Switch makes your VMs “ride along” with the host’s network permissions. They’re hidden behind NAT, which is why your firewall only ever sees the host’s traffic.

Additional Information

If you want to apply a VLAN tag to a physical NIC that’s not a Hyper-V host on Windows:

Since I didn’t see anything in the usual Network  IPv4 configuration window in Windows. I decided to see how useful this AI stuff is.. so I asked it…

“how do I VLAN tag the IP on windows 11 interface?”

Quick Answer: On Windows 11, VLAN tagging isn’t enabled by default. You need a NIC that supports IEEE 802.1Q VLAN tagging, then configure the VLAN ID either through the NIC’s Device Manager properties (Advanced tab → VLAN ID) or by using Intel PROSet/ANS tools or PowerShell (Set-NetAdapter -VlanID) if your driver supports it.

🖥️ Steps to Configure VLAN Tagging on Windows 11

1. Check NIC Compatibility

  • VLAN tagging requires a network card (NIC) that supports IEEE 802.1Q.
  • Not all NICs expose VLAN options in Windows. Intel and Broadcom NICs often support it, but you may need vendor-specific drivers.

2. Configure via Device Manager

  • Press Win + R, type devmgmt.msc, and hit Enter.
  • Expand Network adapters, right-click your NIC → Properties.
  • Go to the Advanced tab.
  • Look for VLAN ID or Priority & VLAN.
  • Enter the VLAN ID (e.g., 100) and click OK.

3. Configure via Intel PROSet/ANS (if supported)

  • Install Intel PROSet/Advanced Network Services (ANS).
  • In the NIC properties, Intel’s tab allows creating multiple VLAN interfaces with different IDs.
  • Each VLAN interface can then be assigned its own IP address.

4. Configure via PowerShell

  • Open PowerShell as Administrator.
  • Run:
    Set-NetAdapter -Name "Ethernet" -VlanID 100
    
  • Replace "Ethernet" with your adapter name and 100 with your VLAN ID.
  • Note: This only works if your NIC driver supports VLAN tagging. Otherwise, you’ll get an error like “Requested operation not supported on adapter”.

5. Assign IP Address

  • Once VLAN tagging is enabled, go to Network & Internet settings → Change adapter options.
  • Right-click the VLAN-enabled adapter → Properties → Internet Protocol Version 4 (TCP/IPv4).
  • Assign the IP address, subnet mask, and gateway for that VLAN.

⚠️ Important Notes

  • Driver Support is Key: If your NIC driver doesn’t expose VLAN options, you cannot tag VLANs directly in Windows without Hyper-V or third-party tools.
  • Hyper-V Alternative: You can create VLAN-tagged virtual NICs using Hyper-V’s Virtual Switch Manager if your hardware doesn’t support native tagging.
  • Testing: Use ping or tracert to verify connectivity once VLAN tagging is applied.

✅ Bottom Line: To VLAN tag an IP on Windows 11, you must enable VLAN tagging on the NIC (via Device Manager, Intel PROSet/ANS, or PowerShell if supported), then assign the IP to that VLAN interface. If your NIC doesn’t support VLAN tagging, you’ll need Hyper-V or vendor-specific utilities.

My response “I’ve got an intle 82579LM Gig NIC, I see it says VLAN enabled in the advanced tab, but I don’t see an area to configure it, I’m sure the NIC supports it since I was using ESXi with vlan tagging just fine.”

That’s a great observation — the Intel 82579LM definitely supports VLAN tagging at the hardware level, but here’s the catch: Windows drivers for that NIC don’t expose VLAN configuration in the same way ESXi or Linux does.

🔎 Why You Don’t See VLAN Config in Windows

  • The Advanced tab in Device Manager shows “Priority & VLAN” or “VLAN Enabled,” but that option only toggles whether the NIC can process VLAN-tagged frames.
  • Unlike ESXi, Windows doesn’t let you directly assign VLAN IDs to a physical NIC unless the driver provides a VLAN interface creation utility (like Intel PROSet/ANS).
  • For the 82579LM, Intel stopped including VLAN configuration in their Windows drivers after Windows 8/10. On Windows 11, the driver exposes VLAN capability but no UI to configure VLAN IDs.

✅ Workarounds

Here are the practical options:

1. Intel PROSet/ANS (if available)

  • Older Intel drivers came with Advanced Network Services (ANS), which let you create VLAN sub-interfaces.
  • On Windows 11, Intel no longer bundles ANS with drivers for the 82579LM, so you won’t see the VLAN tab.

2. PowerShell Command

  • Windows has a Set-NetAdapter -VlanID cmdlet, but it only works if the NIC driver supports VLAN tagging through the OS.
  • On the 82579LM with current Windows 11 drivers, this usually fails with “Requested operation not supported”.

3. Hyper-V Virtual Switch

  • If you enable Hyper-V, you can create a virtual NIC and assign VLAN IDs there.
  • This is the most reliable way to do VLAN tagging on Windows when the physical NIC driver doesn’t expose VLAN config.

4. Third-Party NIC Utilities

  • Some OEMs (Dell, HP) ship custom drivers/utilities that re-enable VLAN tagging. If your system vendor has one, that’s worth checking.

⚠️ Bottom Line

Your Intel 82579LM can tag VLANs, but on Windows 11 the Intel driver doesn’t expose VLAN configuration. That’s why you don’t see a place to set the VLAN ID.

  • On ESXi/Linux → VLAN tagging works natively.
  • On Windows → You need Intel PROSet/ANS (no longer supported for this NIC), Hyper-V, or vendor-specific drivers.

Well ok I guess once we install Hyper-V hopefully it’ll be more like ESXi in a sense and we can tag.

*UPDATE* ExternalSwitch, WiFi NIC

I was trying to use a Hyper-V server with an External Switch, bound to a WiFi NIC. and was getting unexpected results based on the above information. but my results were NOT as I had expected, I was assuming Default Switch behavior. You may be asking, “Why would you expect that behavior if you’re creating an External Switch?”  Now, if you read my Wireless ESXi host blog, you probably are well aware of the VLAN limitations of WiFi, and can never truly be used as a trunk port (Limitation of the 802 standard, not of OS or software).

So how could a ExternalSwitch work, via WiFi if the layer 2 broadcast doesn’t work and can’t “speak” with the rest of the layer 2 stack? Yet I create a VM and it DOES get a DHCP lease address from my local subent?! What the heck is going on here…

So I had to ask AI, what was going on here, it says, yeah… that’s expected… here’s the deets… get ready.. it’s a long one….

BAH-LETED, there was way tooooo much BS from the AI response to justify keeping this data in here… long story short… Local subnet VMs work fine (it does ARP Masquerading), VLANs will never work per the usual BS I’ve talked about in my Wireless ESXi host blog.

Careful Cloning ESXi Hosts

I’ll keep this post short. I was doing some ESXi host deployments in my home lab, and I noticed that when I would install on a 120GB SSD, the install would go smoothly, but I wasn’t able to use any of the storage as a Datastore. However, if I took a fresh install copy of ESXi from installing onto an 8GB USB Stick and DD’d it to the 120GB SSD I got several advantages from this:

  1. When done via a USB3 Pipe of Linux live holding a copy of my base image to deploy I could get speeds in excess of 100 MB/s, and with only 8GB of data to transfer, the “install” would complete in a mere 90 seconds.
  2. The IP address and root password are preconfigured to what I already now, and I can simply change the IP address from the DCUI and call it a day.

Using this method I could have a host up in less than 5 minutes (2 min to boot linux live, 90 seconds to install the base ESXi OS image, and 2 more to boot ESXi). This was of course on machine without ECC and all the server hardware firmware jazz… in those cases install times are always longer. anyway…

This was an amazing option, until I noticed that when I connect one machine in I just deployed and changed the IP address, and (since I’m super anal about networking during this type of project/operations) I noticed my ping from one machine (a completely different IP address) started to drop when the new device came up… and after a while the ping responses would come back but drop from the new host, and vice versa, flip and flop it goes. I’m used to this usually if there’s an IP conflict and two devices have the same IP address. In this case they were different IP addresses… after enough symptom gathering and logical deduction of because I had to assume that the MAC address just be the same and this is the same problem in reverse (different IP’s but same MAC) and as such experiencing the same symptoms.

To validate this I simply deployed my image to a new machine, then I went on the hunt to figure out how to see the MAC address, since I couldn’t plug in the NIC and get to the web based MGMT interface I had to figure out how to do that via the console CLI directly… mhmm after enough googling on my phone I found this spiceworks thread with my answer:

vim-cmd hostsvc/net/info | grep “mac =”

I then checked this against the ESXi host that I saw the flipping flopping with, and sure enough they matched…  After doing a fresh install I noticed that the first 3 sections match the physical MAC, but in my DD deployed ones they retain the MAC of the system from which it was installed and those when I ran the command above, I could tell which ones were deployed via my method. This was further mentioned in this reddit thread by a commenter who goes by the name of sryan2K1:

“The physical NIC MACs are never used. vmk ports, along with VMs themselves will all use VMWare’s OUI as the first half of the address on the wire.”

OK, now maybe I can still salvage my deployment method by simply deleting and recreating the VMK after deployment, but I’d guess it best be done via the DCUI or direct console… I found one KB by VMware/Broadcom but it gave a 404 but Luckly there was a wayback machine link for it here.

Which states the following:

“During Initial Installation and DCUI, ESXi management interface (default vmk0) is created during installation.

The MAC address assigned will be the primary active physical NIC (pnic) associated.

If the associated vmnic is modified with the management interface vmkernel will once again assign MAC address of the associated physical NIC.

To create a VMkernel port and attach it to a portgroup on a Standard vSwitch, run these commands:

esxcli network ip interface add --interface-name=vmkX --portgroup-name=portgroup
esxcli network ip interface ipv4 set --interface-name=vmkX --ipv4=ipaddress --netmask=netmask --type=static"

Alternatively, you can also use esxcli to create the management interface vmkernel on the VDS.

Creation of the management interface with the ‘esxcli network’ will generate a VMware Universally Unique address instead of the pnic MAC address.

It is recommended to use the esxcli network IP interface method to create the management interface and not use DCUI.

Workarounds:               None

Additional Information:
Using DCUI to remove vmnic binding from management vmkernel or any modification will apply change at vSwitch level. Management interface is associated with propagating the change to any port groups within the vSwtich level.

Impact/Risks:                None.”

I”m assuming it means if you use the DCUI to reconfigure the MGMT interface settings the MAC will automatically be reconfigured to match what I found during initial clean install and mentioned in the reddit thread of using the first 3 sections to derive the MAC of the VMK.

But what if you don’t have any additional interfaces to use to make the section change in the DCUI to have that actually happen? cause what I’ve noticed changing the IP address and disabling IPv6 and rebooting did not change the VMK’s MAC address. Oh there’s in option in the DCUI “Reset Network Settings” within there there’s several options, I simply picked reset to factory defaults. Said success, checked the MAC via the first command stated above and bam the VMK nic changed to what it should be! Sweet my deployment method is still viable.

Hope this helps someone.

Wireless ESXi Host

The Story

So, the other day I pondered an idea. I wanted to start making some special art pieces made from old motherboards, and then I also started to wonder could I actually make such an art piece… and have it functional?

I took an apart my old build that was a 1U server I made from an old PA-500 and a motherboard I repurposed from a colleague who gifted me their old broken system. Since it was a 1U system, I had purchased 2 special pieces to make it work, a special CPU heatsink (complete solid copper, with a side blower fan, and a 300 watt 1U PSU. both of which made lots of noise.

I also have another project going called “Operation Shut the fuck up” in which all the noisy servers I run will be either shutdown or modified to make zero noise. I hope with the project to also reduce my overall power consumption.

So I started by simply benching the Mobo and working off that, which spurred a whole interest into open case computer designs. I managed to find some projects on Thingiverse for 2020 extrusions and corner braces, cable ties… the works. The build was coming along swimmingly. There was just one thing that kept bugging me about the build… The wires…

Now I know the power cable will be required reguardless, but my hope was to have/install an outlet at the level the art piece was going to be placed at and have it nicely nested behind the art piece to hide it. Now there were a couple ways to resolve this.

  1. Use an Ethernet over Power (Powerline) adapter to use the existing copper power lines already installed in the house. (Not to be confused with PoE).
    There was just one problem with this, my existing Powerline kit died right when I wanted to use it for the purpose. (Looking inside looks like the, soldered to the board, fuse blew, might be as simple as replacing that but it could be a component behind the fuse failed and replacing it would simply blow the new fuse).
    *This is still a very solid option as the default physical port can be used and no other software/configuration/hackery needs to be done, (Plug n Play).
  2.  The next best option would be to use one of these RJ45 to Wireless adapters:
    Wireless Portable WiFi Repeater/Bridge/AP Modes, VONETS VAP11G-300.
    VONETS VAP11G-500S Industrial 2.4GHz Mini WiFi Bridge Wireless Repeater/Router Ethernet to WiFi Adapter
    This option is not as good as the signal quality over wireless is not has good as physical even when using Powerline adapters. However, this option much like the Powerline option, again allows the use of the default NIC, and only the device itself would need to be preconfigured using another system but otherwise again no software/configuration/hackery needs to be done.
  3.  Straight up use a WiFi Adapter on the ESXi host.

Now if you look up this option you’ll see many different responses from:

  1. It can’t be done at all. But USB NICs have community drivers.
    This is true and I’ve used it for ESXi hosts that didn’t have enough NICs for the different Networks that were available (And VLAN was not a viable option for the network design). But I digress here, that’s not what were are after, Wifi ESXi, yes?
  2.  It can’t be done. But option 1, powerline is mentioned, as well as option 2 to use a WiFi bridge to connect to the physical port.
  3.  Can’t be done, use a bridge. Option 2 specified above. and finally…
  4.  Yeah, ESXi doesn’t support Wifi (as mentioned many times) but….. If you pass the WiFi hardware to a VM, then use the vSwitching on the host.. Maybe…

As directly quoted by.. “deleted” – “I mean….if you can find a wifi card that capable, or you make a VM such as pfsense that has a wifi card passed through and that has drivers and then you router all traffic through some internal NIC thats connected to pfsense….”

It was this guys comment that I ran with this crazy idea to see if it could be done…. Spoiler alert, yes that’s why I’m writing this blog post.

The Tasks

The Caveats

While going through this project I was hit with one pretty big hiccup which really sucks but I was able to work past it. That is… It won’t be possible to Bridge the WAN/LAN network segments in OPNsense/PFsense with this setup. Which really sucked that I had to find this out the hard way… as mentioned by pfsense parent company here:

“BSS and IBSS wireless and Bridging

Due to the way wireless works in BSS mode (Basic Service Set, client mode) and IBSS mode (Independent Basic Service Set, Ad-Hoc mode), and the way bridging works, a wireless interface cannot be bridged in BSS or IBSS mode. Every device connected to a wireless card in BSS or IBSS mode must present the same MAC address. With bridging, the MAC address passed is the actual MAC of the connected device. This is normally a desirable facet of how bridging works. With wireless, the only way this can function is if all the devices behind that wireless card present the same MAC address on the wireless network. This is explained in depth by noted wireless expert Jim Thompson in a mailing list post.

As one example, when VMware Player, Workstation, or Server is configured to bridge to a wireless interface, it automatically translates the MAC address to that of the wireless card. Because there is no way to translate a MAC address in FreeBSD, and because of the way bridging in FreeBSD works, it is difficult to provide any workarounds similar to what VMware offers. At some point pfSense® software may support this, but it is not currently on the roadmap.”

Cool what does that mean? It means that if you are running a flat /24 network, as most people in home networks run a private subnet of 192.168.0.0/24, that this device will not be able to communicate in the layer 2 broadcast domain. The good news is ESXi doesn’t needs to work, or utilizes features of broadcast domains. It does however mean that we will need to manage routes as communications to the host using this method will have to be on it’s own dedicated subnet and be routed accordingly based on your network infrastructure. If you have no idea what I’m talking about here then it’s probably best not to continue on with this blog post.

Let’s get started. Oh another thing, at the time of this writing a physical port is still required to get this setup as lots of initial configurations still need to take place on the ESXi host via the Web GUI which can initially only be accessible via the physical port, maybe when I’m done I can make a mirco image of the ESXi hdd with the required VM, but even then the passthrough would have to be configured… ignore this rambling I’m just thinking stupid things…

Step 1) Have a ESXi host with a PCI-e based WiFi card.

I’ve tested this with both desktop Mobo with a PCI-e Wifi card, and a laptop with a built in Wifi Card, in both cases this process worked.

As you can see here I have a very basic ESXi server with some old hardware but otherwise still perfectly useable. For this setup it will be ESXi on USB stick, and for fun I made a Datastore on the remaining space on the USB stick since it was a 64 Gig stick. This is generally a bad idea, again for the same reasons mentioned above that USB sticks are not good at HIGH random I/O, and persistent I/O on top of that, but since this whole blog post is getting an ESXi host managed via WiFi which is also frowned upon why not just go the extra mile and really piss everyone off.

Again I could have done everything on the existing SATA based SSD and avoid so much potential future issue…. but here I am… anyway…

You may also note that at this time in the post I am connecting to a physical adapter on the ESXi host as noted by the IP addresses… once complete these IP addresses will not be used but remain bound the physical NIC.

Step 2) Create VM to manage the WiFi.

Again I’m choosing to use OPNsense cause they are awesome in my opinion.

I found I was able to get away with 1 GB of memory (even though min stated is 2) and 16 GB HDD, if I tried 8 GB the OPNsense installer would fail even though it states to be able to install on 4 GB SD Cards.

Also note I manually change boot from BIOS to EFI which has long been supported. At this stage also check off boot into EFI menu, this allows the VMRC tool to connect to ISO images from my desktop machine that I’m using to manage the ESXi host at this time.

Installing OPNsense

Now this would be much faster had I simply used the SSD, but since I’m doing everything the dumbest way possible, the max speed here will be roughly 8 MB/s… I know this from the extensive testing I’ve done on these USB drives from the ESXi install. (The install caused me so much grief hahah).

Wow 22 MB/s amazing, just remember though that this will be the HDD for just the OPNsense server that won’t need storage I/O, it’ll simply boot and manage the traffic over the WiFi card.

And much like how ESXi installed on the exact same USB drive, we are going to configure OPNsense to not burn out the drive. By following the suggestions in this thread.

Configuring  OPNsense

Much like the ESXi host itself at this point I have this VM connected to the same VMPG that connects to my flat 192.168 network. This will allow us to gain access to the web interface to configure the OPNsense server exactly in the same manner we are currently configuring the ESXi host. However, for some reason the main interface while it will default assign to LAN it won’t be configured for DHCP and assumes 192.168.1.1/24 IP… cool, so log into the console and configure the LAN IP address to be reachable per your config, in my case I’m going to give it an IP address in my 192.168.0.0/24 network.

Again this IP will be temporary to configure the VM via the Web GUI. Technically the next couple steps can be done via the CLI but this is just a preference for me at this time, if you know what you are doing feel free to configure these steps as you see fit.

I’m in! At this point I configure SSH access and allow root and password login. Since this it a WiFi bridged VM and not one acting as a firewall between my private network and the public facing internet this is fine for me and allows more management access. Change these how you see fit.

At this point, I skip the GUI wizard.  Then configured the settings per the link above.

Even with only 1 GB of memory defined for the VM, I wonder if this will cause any issues, reboot, system seems to have come up fine… moving on.

Holy crap we finally have the pre-reqs in place. All we have to do now is configure the WiFi card for PCI passthrough, give it to the VM, and reconfigure the network stacks. Let’s go!

Locate WiFi card and Configure Passthrough

So back on the ESXi web interface go to … Host -> Manage -> Hardware and configure the device for pasththrough until, you find all devices are greyed out? What the… I’ve done this 3 times what happed….

All PCI Passthrough devices grayed out on ESXi 6.7U3 : r/vmware (reddit.com)

FFS, OK I should have mentioned this in the pre-reqs but I guess in all my previous builds test this setting must have been enabled and available on the boards I was using… I hope I’m not hooped here yet again in this dang project…

Great went into the BIOS could find nothing specific for VT-d or VT-x (kind of amazed VM were working on this thing the whole time. I found one option  called XD bit or something, it was enabled, I changed it to disabled, and it caused the system to go into a boot loop. It would start the ESXi boot up and then half way in randomly reboot, I changed the setting back and it works just fine again.

I’m trying super hard right now not to get angry cause everything I have tried to get this server up and running while not having to use the physical NIC has failed… even though I know it’s possible cause I did this 2 other times successfully and now I’m hung cause of another STUPID ****ING technicality.

K I have one other dumb idea up my ass… I have a USB based WiFi NIC, maybe just maybe I can pass that to OPNsense…

VMware seems to possibly allow it: Add USB Devices from an ESXi Host to a Virtual Machine (vmware.com)

OPNsense… Maybe? compatible USB Wifi (opnsense.org)

Here goes my last and final attempt at this hardware….

Attempting USB WiFi Passthrough

Add device, USB Controller 2.0.

Add Device, Find USB device on host from drop down menu.

Boot VM….. (my hearts racing right now, cause I’m in a HAB (Heightened Anger Baseline) and I have no idea if this final work around is going to work or not).

Damn it doesn’t seem to be showing under interfaces… checking dmesg on the shell…

I mean it there’s it has the same name as the PCI-e based WiFi card I was trying to use, but that is 1) pulled from the machine, and 2) we couldn’t pass it through, and dmesg shows it’s on the usbus1… that has to be it… but why can’t I see it in the OPNsense GUI?

OMG… I think this worked… I went to Interfaces wireless, then added the run0 I saw in dmesg….

I then added as an available interface….

For some weird reason it gave it a weird assignment as WifIBridge… I went back into the console and selected option 2 to assign interfaces:

Yay now I can see an assignable interface to WAN. I pick run0

Now back into OPNsense GUI… OMG… there we go I think we can move forward!

Once you see this we can FINALLY start to configure the wireless connection that will drive this whole design! Time for a quick break.

Configuring WiFi on OPNsense

No matter if you did PCI-e passthrough or USB passthrough you should now have an accessible OPNsense via LAN, and assigned the WiFi device interface to WAN. Now we need to get WAN connected to the actual WiFi.

So… Step 1) remove all blocking options to prevent any network issues, again this is an internal bridge/router, and not a Edge Firewall/NAT.

Uncheck Block Private Networks (Since we will be assigning the WAN interface a Private IP), and uncheck Block bogon networks.

Step 2) Define your IP info. In my case I’m going to be providing it a Static IP. I want to give it the one that is currently being used to access it that is bound to the vNIC, but since it’s alread bound and in use we’ll give it another IP in the same subnet and move the IP once it’s released from the other interface. For now we will also leave it as a slash 32 to prevent a network overlap of the interface bound on LAN thats configured for a /24.

No IPv6.

Step 3) Define SSID to connect to and Password.

I did this and clicked apply and to my dismay.. I couldn’t get a ping response… I ssh’d into the device by the current VMX nic IP and even the device itself couldn’t ping it (interface is down, something is wrong).

Checking the OPNsense GUI under INterface Assignments I noticed 2 WiFI interfaces (somehow I guess from me creating it above, and then running the wizard on the console?).

Dang I wanted to grab a snip, but from picking the main one (the other one was called a clone), it has now been removed from the dropdown, and after picking that one the pings started working!

Not sure what to say here, but now at this point you should have a OPNsnese server accessible by LAN (192.168.0.x) and WAN (192.168.0.x). The next thing is we need to make the Web interface accessible by the WAN (Wireless) interface.

Basically, something as horrendous as this drawing here:

Anyway… the first goal is to see if the WiFi hold up, to test this I simply unplug the physical cable from the beaitful diagram above, and make sure the pings to the WAN interface stay up… and they both went down….

This happened to me on my first go around on testing this setup… I know I fixed it.. I just can’t remember how… maybe a reboot of the VM, replug in physical cable. Before I reboot this device I’ll configure a gateway as well.

Interesting, so yup that fixed the WiFi issue, OPNsense now came up clean and WiFi still ping response even when physical nic is removed from the ESXi host… we are gonna make it!

interesting the LAN IP did not come up and disappeared. But that’s OK cause I can access the Web GUI via the WAN IP (Wirelessly).

finally OK, we finally have our wireless connection, now we just need to create a new vSwitch and MGMT network on the ESXi host that we will connect to the OPNsense on the VMX0 side (LAN) that you can see is free to reconfigure. This also free’d the IP address I wanted to use for the WAN, but since I’ve had so many issues… I’m just going to keep the one I got working and move on.

Configure the Special Managment network.

I’m going to go on record and say I’m doing it this way simply cause I got this way to work, if you can make it work by using the existing vSwitch and MGMT interfaces, by all means giver! I’m keeping my existing IPs and MGMT interfaces on the default switch0 and creating a new one for the wireless connection simply so that if I want to physically connect to the existing connection.. I simply plug in the cable.

Having said that on the ESXi host it’s time to create a new vSwitch:

Now create the new VMK, the IP given here is the in the new subnet that will be routed behind the OPNsense WAN. In my example I created a new subnet 192.168.68.0/24 this will be routed to the WAN IP address given to OPNsense in my example here that will be 192.168.0.33. (Outside the scope of this blog post I have created routes for this on my devices gateway devices, also since my machine is in the same subnet at the OPNsense WAN IP, but the OPNsense WAN IP address is not my subnets gateway IP this can cause what is known as asymetric routing, to resolve this you simply have to add the same route I just mentioned to the machine managing the devices. You have been warned, design your stuff better than I’m doing here… this is all simply for educational purposes… don’t do this ever in production)

Now we need to create a VMPG for the VM to connect the VMX0 IP into the new vSwitch to provide it the gateway IP for that new subnet (192.168.68.1/24)

Now we can finally configure the vNIC on the OPNsense VM to this new VMPG:

Before we configure the OPNsense box to have this new IP address let’s configure the ESXi gateway to be that:

OK finally back on the OPNsense side let’s configure the IP address…

Now to validate this it should simply be making sure the ESXi host can ping this IP…

All I should have to do now is configure the route on my machine doing all this work and I should also be able to ping it…

More success… final step.. unplug physical nic to pings stay up?? OMG and they do!!! hahaha:

As you can see the physical NIC IP drops but the new secret MGMT IPs behind the WiFi stay up! There’s one final thing we need to do though.

Configure Auto Start of OPNsense

This is a critical step in the design setup as the OPNsense needs to come up automatically in order to be able to manage the ESXi host if there is ever a reboot of the host.

Then simply configure the auto start setting for this VM:

I also go in and change the auto start delay to 30 seconds.

Summary

And there you have it… and ESXi host completely managed via WiFi….

There are a ton of limitations:

  1. No Bridging so you can’t keep a flat layer 2 broadcast domain. Thus:
  2. Requires dedicated routes and complex networking.
  3. All VM traffic is best handled directly on internal vSwitch otherwise all other VM traffic will share the same WiFi gateway providing a terrible experince.
  4. The Web interface will become sluggish when the network interface is under load.
  5.  However it is overall actually possible.
  6. * Using PCI-e passthrough disallows snapshots/vMotions of the OPNsense VM but USB does allow it, when doing a storage vMotion the VM crashed on me, for some reason auto start disabled too had to manually start the VM back up. (I did this by re-IPing the ESXi server via console and plugging in a phsyical cable)
  7. With USB WiFi Nic connections can be connected/disconnected from the host, but with PCI-e Passthrough these options are disabled.
  8. With USB NIC you can add more vNICs to OPNsense and configure them, it just brings down the network overall for about 4-5 min, but be patient it does work.Here’s a Speedtest from a Windows Virtual Machine on the ESXi host.

Hope you all enjoyed this blog post. See ya all next time!

*UPDATE* Remember when I stated I wanted to keep those VMKs in place incase I ever wanted to plug the physical cable back in? Yeah that burnt me pretty hard. If you want a backup physical IP make it something different then you existing network subets and write it down on the NIC…

For some really strange reason HTTPS would work but all other connections such as SSH would timeout very similar to an asymmetric routing issue, and it actually cause it kind was. I’m kinda shocked that HTTPS even managed to work… huh…

Here’s a conversation I had with other on VMware IRC channel trying to troubleshoot the issue. Man I felt so dumb when I finally figured out what was going on.

*Update 2* I notice that the CPU usage on the OPNsense VM would be very high when traffic through it was taking place (and not even high bandwidth here either) AND with the pffilter service disabled, meaning it working it pure routing mode.

High CPU load with 600Mbit (opnsense.org)

Poor speeds and high CPU usage when going through OPNsense?

“Furthermore, set the CPU to 1 core and 4 sockets. Make sure you use VirtIO nics and set Multiqueue to 4 or 8. There is some debate going on if it should be 4 or 8. By my understanding, setting it to 4 will force the amount of queues to 4, which in this case matches your amount of CPU cores. Setting it to 8 will make OPNsense/FreeBSD select the correct amount.” Says Mars

“In this case this is also comparing a linux-based router to a BSD based one. Linux will be able to scale throughput much easily with less CPU power required when compared to the available BSD-based routers. Hopefully with FreeBSD 13 we’ll see more optimization in this regard and maybe close the gap a bit compared to what Linux can do.” Says opnfwb

Mhmmm ok I guess first thing I can try is upping the CPU core count. But this VM also hosts the connection I need to manage it… Seems others have hit this problem too…

Can you add CPU cores to VM at next restart? : r/vmware (reddit.com)

while the script is decent, the comment by cowherd is exactly what I was thinking I was going to do here: “Could you clone the firewall, add cores to the clone, then start it powering up and immediately hard power off the original?”

I’ll test this out when time permits and hopefully provide some charts and stats.

PA VM in bazaar state… by Design

So today I had some weird stuff happening (Fedora Download was downloading slow, 300 KB/s)… I thought it was the mirror, but no matter what mirror I picked I had the same results, I asked a buddy to verify my findings and they could download Fedora with speed… Long story short, I thought maybe it was my firewall, and my colleague mentioned the same. Since this is a Lab setup it would be nice to get a perpetual license for learning purposes, but PAN clearly don’t work like. I was pretty sure my license had expired, so decided to first quick finds out what happens when a license expires: What Happens When Licenses Expire? (paloaltonetworks.com)…

Threat Prevention
Alerts appear in the System Log indicating that the license has expired.
You can still:
  • Use signatures that were installed at the time the license expired, unless you install a new Applications-only content update either manually or as part of an automatic schedule. If you do, the update will delete your existing threat signatures and you will no longer receive protection against them.
  • Use and modify Custom App-ID™ and threat signatures.
You can no longer:
  • Install new signatures.
  • Roll signatures back to previous versions.

Good to know, nothing that would cause the issue I’m experiencing….

DNS Security
You can still:
  • Use local DNS signatures if you have an active Threat Prevention license.
You can no longer:
  • Get new DNS signatures.

nope… and…

Advanced URL Filtering / URL Filtering
You can still:
  • Enforce policy using custom URL categories.
You can no longer:
  • Get updates to cached PAN-DB categories.
  • Connect to the PAN-DB URL filtering database.
  • Get PAN-DB URL categories.
  • Analyze URL requests in real-time using advanced URL filtering.
WildFire
You can still:
  • Forward PEs for analysis.
  • Get signature updates every 24-48 hours if you have an active Threat Prevention subscription.
You can no longer:
  • Get five-minute updates through the WildFire public and private clouds.
  • Forward advanced file types such as APKs, Flash files, PDFs, Microsoft Office files, Java Applets, Java files (.jar and .class), and HTTP/HTTPS email links contained in SMTP and POP3 email messages.
AutoFocus
You can still:
  • Use an external dynamic list with AutoFocus data for a grace period of three months.
You can no longer:
  • Access the AutoFocus portal.
Cortex Data Lake
You can still:
  • Store log data for a 30-day grace period, after which it is deleted.
  • Forward logs to Cortex Data Lake until the end of the 30-day grace period.
GlobalProtect
You can still:
  • Use the app for endpoints running Windows and macOS.
  • Configure single or multiple internal/external gateways.
You can no longer:
  • Access the Linux OS app and mobile app for iOS, Android, Chrome OS, and Windows 10 UWP.
  • Use IPv6 for external gateways.
  • Run HIP checks.
  • Enforce split tunneling based on destination domain, client process, and video streaming application.

All a bunch of nope…

VM-Series
Support
You can no longer:
  • Receive software updates.
  • Download VM images.
  • Benefit from technical support.

This is a VM series yes… so what does that link mean….

VM-Series
You can still:
You can continue to configure and use the firewall you deployed prior to the license expiring with no change in session capacity. The firewall won’t reboot automatically and cause a disruption in traffic.
However, if the firewall reboots for any reason, the firewall enters an unlicensed state. While unlicensed, a firewall supports a maximum of 1,200 sessions. No other management plane features or configuration options are restricted.

OK… Maybe… but I’m sure a download of a single file doesn’t take over 1,200 sessions… while I did reboot the unit (cloned, power off OG, power on clone, etc)

All other things are the same as posted above… Then I noticed some really weird things….

  1. Checking for updates doesn’t state anything about license status, just tries and quietly fails.
  2. Checking support status shows “Device not found on this update server”
  3. Dynamic Updates do not show a “currently installed” version.
    1. The current version installed with Review Policies, and review apps under action.
    2. The previous installed one will have the same plus a revert action.
    3. Downloaded one will have an install action.
    4. All others seen since last communication to PAN will have download
  4. Retrieving licenses from licenses server returns “Failed to install features. The device is not found.
  5. Finally the smoking gun… Serial Number on the Dashboard will be listed as unknown.

So, I ended Googling this and found not one, but TWO KB’s!!!

Serial number becomes “unknown” after changing the instance typ… – Knowledge Base – Palo Alto Networks

and

Serial number becomes “unknown” upon rebooting PA-VM – Knowledge Base – Palo Alto Networks

After reading these, it all made sense… and it’s all rather dumb… to paraphrase it simply….

It’s due to DRM, how the DRM works is it derives the serial number from two ID’s CPUID and UUID… and when you migrate a PAN VM the CPU is different cause of the different host it resides… this in turn breaks the licensing.

*Standing Ovation*

What’s PAN solution… Open a support ticket… that’s right.. instead of coming up with a technical solution to make DRM work while still retaining the ability to migrate the VM (The most important and valuable reason why you want to run it as a VM anyway)….

Instead of having a way to edit the CPUID and UUID in the PAN portal to fix this yourself…..

No they want you to waste their tech support personals time….

This ….. IS……. DUMB!!!!!