{"id":1823,"date":"2026-08-24T20:31:47","date_gmt":"2026-08-25T01:31:47","guid":{"rendered":"https:\/\/zewwy.ca\/?p=1823"},"modified":"2026-08-24T20:31:47","modified_gmt":"2026-08-25T01:31:47","slug":"setting-up-a-proxmox-host","status":"publish","type":"post","link":"https:\/\/zewwy.ca\/index.php\/2026\/08\/24\/setting-up-a-proxmox-host\/","title":{"rendered":"Setting up a Proxmox host"},"content":{"rendered":"

<\/span>Install Proxmox<\/span><\/h1>\n

<\/span>Hardware<\/span><\/h2>\n

Step 1) Pick Hardware. Important is CPU support. Mostly ARM or x86_64.
\n– My host a HPE DL160 G9.<\/p>\n

<\/span>Software<\/span><\/h2>\n

Step 2) Install using appropriate installer image<\/a>.
\n– In my case x86_64 version 9.2
\n– I installed to an internal 32GB sd card.<\/p>\n

<\/span>Storage (Physical)<\/span><\/h2>\n

Step 3) VM Storage.
\nI’ve discussed this in the past<\/a> specially when it comes to shared storage options. There you can see a picture of all the options available to Proxmox, and if it supports snapshots or if its shared. For ease sake of this post we’re going to stick to local storage.\u00a0 With the minted information from that chart alone ZFS would seem the winner, however….<\/p>\n

Choosing the right storage architecture for a virtualization host requires a careful balance between resource allocation, hardware capabilities, and performance goals. For this build\u2014featuring 50 GB of memory, an HPE B140i controller running in SATA AHCI pass-through mode, and a mix of SSDs and a mechanical drive\u2014maximizing raw performance and preserving system RAM for virtual machines is our primary objective. By selecting LVM-Thin instead of ZFS, we bypass the heavy computational and memory overhead of a Copy-on-Write filesystem, ensuring that nearly all 50 GB of RAM remains dedicated strictly to our workloads. The design stripes multiple solid-state drives into a single, high-performance LVM-Thin volume group to multiply IOPS and throughput for VM boot disks. Meanwhile, the standalone 4TB spindle drive is formatted as a standard, zero-RAM-footprint Linux directory to act as an isolated target for possible Proxmox backups and static file shares. This hybrid, LVM-centric approach eliminates storage controller bottlenecks, maximizes the lifespan and speed of our SSDs, and relies on a robust backup strategy rather than restrictive hardware or software redundancy.<\/p>\n

After messing around about an hour, I found out the reason I wasn’t seeing the drives was due to a controller configuration (it was already set to Sata AHCI support mode), which I double verified by seeing the disks and running dd commands against them to get sequential performance numbers. The reason, was cause apparently in this mode drives are not hot swapable<\/a>.<\/p>\n

Could attempt a manual rescan via the shell backend, I guess but as noted there. “If your SATA controller supports hot swap, it should “just work(tm).”<\/a><\/p>\n

<rant> Stupid ass HP, always causing me to waste my life away cause of their stupid ass storage controller and firmware\/driver choices.. ughhh <\/rant><\/p>\n

<\/span>Turn off Swap<\/span><\/h2>\n

SIDE QUEST! Congratulations you just entered a side quest on your way to configuring your storage for your PVE hypervisor. SWAP!<\/p>\n

An critical optimization step for any Proxmox host booting from flash media is managing the system’s swap space. By default, the Debian-based Proxmox installer creates a virtual memory swap partition directly on the boot drive. When running Proxmox from an internal SD card, leaving swap enabled is a hardware hazard; Linux will continuously shift idle processes onto the card, exhausting its low write-endurance and risking boot environment corruption. Because this host boasts a healthy 50 GB of physical RAM, we immediately disabled and removed the default swap volume to shield the SD card from unnecessary wear. However, completely lacking a swap space can lead to kernel instability under unexpected memory spikes. Our strategy resolves this by re-establishing a dedicated swap space directly on our new solid-state storage tier. Crucially, this swap will not be placed inside the dynamic LVM-Thin pool\u2014which can cause file system deadlocks and severe latency\u2014but will instead be carved out as a fixed, pre-allocated ‘Thick’ LVM volume. This hybrid approach ensures the SD card remains read-heavy and protected, while giving the host an ultra-fast, safe SSD safety net without sacrificing valuable system memory to ZFS.<\/p>\n

<\/span>1. Turn off active swap immediately<\/strong><\/span><\/h3>\n
<\/div>\n
\n
\n
\n
\n
bash<\/div>\n<\/div>\n
\n
swapoff -v \/dev\/mapper\/pve-swap\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
\n
<\/span>2. Stop it from turning back on when you reboot<\/strong><\/span><\/h3>\n<\/div>\n<\/div>\n<\/div>\n
Open your filesystem table:<\/div>\n
<\/div>\n
\n
\n
\n
\n
bash<\/div>\n<\/div>\n
\n
nano \/etc\/fstab\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
\n
Look for the line that mentions <\/span>pve-swap<\/code>. It will look similar to this:<\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n
<\/div>\n
\n
\n
\n
\n
text<\/div>\n<\/div>\n
\n
\/dev\/pve\/swap none swap sw 0 0\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
\n
Add a <\/span>#<\/code> at the very beginning of that line to comment it out and disable it permanently:<\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
\n
\n
# \/dev\/pve\/swap none swap sw 0 0\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
\n
<\/span>3. Delete the volume entirely (Optional but recommended)<\/strong><\/span><\/h3>\n<\/div>\n<\/div>\n<\/div>\n
To ensure the OS never touches it again, remove the logical volume entirely:<\/div>\n
<\/div>\n
\n
\n
\n
\n
bash<\/div>\n<\/div>\n
\n
lvremove \/dev\/pve\/swap\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
<\/div>\n<\/div>\n<\/div>\n
Now with swap off we can finally build our LVM groups and move the swap to the SSDs.<\/div>\n
<\/div>\n
<\/span>Back to Storage (Logical)<\/span><\/h2>\n
\n
Step 1: Create the Physical Volumes (PV)<\/strong><\/p>\n
First, we tell LVM that these three specific SSDs are ready to be used as raw storage building blocks.<\/div>\n
\n
\n
\n
\n
pvcreate \/dev\/sda \/dev\/sdb \/dev\/sdc\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
\n
Expected output: Physical volume \"\/dev\/sda\" successfully created. x3<\/code><\/em><\/div>\n<\/div>\n<\/div>\n<\/div>\n
<\/span>Step 2: Combine them into a Volume Group (VG)<\/strong><\/span><\/h3>\n
Now, we pool those three independent drives into one large, unified storage pool. We will name this group pve-fast<\/code><\/strong>.<\/div>\n
\n
\n
\n
\n
vgcreate pve-fast \/dev\/sda \/dev\/sdb \/dev\/sdc\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
\n
Expected output: Volume group \"pve-fast\" successfully created<\/code>.<\/em><\/div>\n<\/div>\n<\/div>\n<\/div>\n
<\/span>Step 3: Verify the Master Pool<\/strong><\/span><\/h3>\n
To confirm everything was combined properly and to check your exact total available space, run:<\/div>\n
\n
\n
\n
\n
bash<\/div>\n<\/div>\n
\n
vgs pve-fast\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
\n
You should see <\/span>pve-fast<\/code> listed with <\/span>3<\/code> physical volumes (<\/span>#PV<\/code>) and a total size that roughly equals the combined capacity of your three SSDs.<\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
<\/div>\n
root@g9-pve:~# pvcreate \/dev\/sda \/dev\/sdb \/dev\/sdc\r\nPhysical volume \"\/dev\/sda\" successfully created.\r\nPhysical volume \"\/dev\/sdb\" successfully created.\r\nPhysical volume \"\/dev\/sdc\" successfully created.\r\nroot@g9-pve:~# vgcreate pve-fast \/dev\/sda \/dev\/sdb \/dev\/sdc\r\nVolume group \"pve-fast\" successfully created\r\nroot@g9-pve:~# vgs pve-fast\r\nVG #PV #LV #SN Attr VSize VFree\r\npve-fast 3 0 0 wz--n- <670.70g <670.70g<\/pre>\n
Creating the Volume Group (VG) only defines the boundaries of your master pool. It tells LVM: “You are allowed to use the storage blocks inside sda, sdb, and sdc.” It does not decide how data is laid out yet.<\/p>\n
The master pool itself is neutral. The choice between Linear or Striped happens entirely in the next step when we create the Logical Volumes (LVs) inside that pool. Why it’s like this, I dunno, I’m just here to figure out how it works.<\/p>\n
<\/span>Back to Swap<\/span><\/h2>\n
We will allocate 4 GB<\/strong> of space for this safety net. We will use the -i 3<\/code> flag to guarantee that any memory swapped to disk is interleaved across all three SSD controllers simultaneously for maximum throughput.<\/div>\n
<\/div>\n
Run these four commands sequentially in your Proxmox CLI:<\/div>\n
<\/div>\n
<\/span>1. Create the Striped Logical Volume<\/strong><\/span><\/h3>\n
We will carve out a new volume named fast-swap<\/code> from your pve-fast<\/code> volume group.<\/div>\n
\n
\n
\n
\n
lvcreate -L <\/span>4<\/span>G -i <\/span>3<\/span> -I <\/span>64<\/span>k -n fast-swap pve-fast\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n<\/div>\n
<\/div>\n
    \n
  • -L 4G<\/code><\/strong>: Allocates exactly 4 Gigabytes of space.<\/span><\/li>\n
  • -i 3<\/code><\/strong>: Forces the volume to stripe data across exactly 3 physical disks (RAID0 behavior).<\/span><\/li>\n
  • -I 64k<\/code><\/strong>: Sets the optimal block stripe size for performance.<\/span><\/li>\n
    <\/ul>\n
    <\/div>\n
    <\/span>2. Format the Volume for Swap<\/strong><\/span><\/h3>\n
    Now we tell the operating system to format this new striped block device specifically as Linux swap space.<\/div>\n
    \n
    \n
    \n
    \n
    mkswap \/dev\/pve-fast\/fast-swap\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
    \n
    <\/span>3. Activate the New Swap Space<\/span><\/strong><\/span><\/h3>\n<\/div>\n<\/div>\n<\/div>\n
    Turn on the newly created SSD swap space right now so the system can begin utilizing it.<\/div>\n
    \n
    \n
    \n
    \n
    swapon \/dev\/pve-fast\/fast-swap\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
    \n
    <\/span>4. Make it Permanent Across Reboots<\/span><\/strong><\/span><\/h3>\n<\/div>\n<\/div>\n<\/div>\n
    We need to register this new location in your system’s filesystem table so it mounts automatically every time the server turns on. Run this command to append the new rule to your configuration file:<\/div>\n
    \n
    \n
    \n
    \n
    echo <\/span>'\/dev\/pve-fast\/fast-swap none swap sw 0 0'<\/span> >> \/etc\/fstab\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
    \n
    <\/span>Verify the Configuration<\/span><\/strong><\/span><\/h3>\n<\/div>\n<\/div>\n<\/div>\n
    To verify that your swap is active, running at top speed, and no longer touching your 32 GB SD card, run:<\/div>\n
    \n
    \n
    \n
    \n
    swapon --show\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
    \n
    You should see <\/span>\/dev\/mapper\/pve--fast-fast--swap<\/code> listed as the sole active device, confirming your SD card is safe.<\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n
    root@g9-pve:~# lvcreate -L 4G -i 3 -I 64k -n fast-swap pve-fast\r\nRounding size 4.00 GiB (1024 extents) up to stripe boundary size <4.01 GiB (1026 extents).\r\nLogical volume \"fast-swap\" created.\r\nroot@g9-pve:~# mkswap \/dev\/pve-fast\/fast-swap\r\nSetting up swapspace version 1, size = 4 GiB (4303351808 bytes)\r\nno label, UUID=a6ec7580-dc9c-40de-8405-b9f8c66a41fd\r\nroot@g9-pve:~# swapon \/dev\/pve-fast\/fast-swap\r\nroot@g9-pve:~# echo '\/dev\/pve-fast\/fast-swap none swap sw 0 0' >> \/etc\/fstab\r\nroot@g9-pve:~# swapon --show\r\nNAME TYPE SIZE USED PRIO\r\n\/dev\/dm-0 partition 4G 0B -1\r\nroot@g9-pve:~# fdisk -l | grep mapper\r\nDisk \/dev\/mapper\/pve-root: 12.8 GiB, 13740539904 bytes, 26836992 sectors\r\nDisk \/dev\/mapper\/pve--fast-fast--swap: 4.01 GiB, 4303355904 bytes, 8404992 sectors<\/pre>\n
    K we’re finally done with swaps. I asked AI why I wasn’t seeing any partitions on the SSDs after allocating them to the LVM pool. it stated:<\/div>\n
    “When we ran the <\/span>pvcreate<\/code> command earlier, LVM took over the raw, unpartitioned block devices (<\/span>sda<\/code>, <\/span>sdb<\/code>, <\/span>sdc<\/code>) directly, rather than writing a traditional partition table like MBR or GPT.<\/span><\/p>\n
    <\/div>\n
    Why LVM Prefers No Partition Layout<\/div>\n
    Tools like fdisk<\/code> and gparted<\/code> look for standard partition tables to tell them where a drive starts and ends. LVM bypasses this entirely:<\/div>\n
      \n
    1. Direct Disk Ownership:<\/strong> LVM wipes out the standard partition header area and writes its own metadata directly to the very first sectors of the raw disk.<\/span><\/li>\n
    2. Simplified Layout:<\/strong> Because we are striping these drives together, creating individual partitions (like sda1<\/code>, sdb1<\/code>) is an unnecessary extra layer of management. By giving LVM the raw disks, it can seamlessly manage the entire drive boundary.<\/span><\/li>\n
      <\/ol>\n
      <\/div>\n
      How to Properly View Your LVM Layout<\/div>\n
      Because fdisk<\/code> doesn’t natively parse LVM metadata structures, it will report that the disks look empty. To see the true reality of your storage, you must use LVM-specific inspection commands instead:<\/div>\n
        \n
      • To see the raw disks LVM owns:<\/strong> pvdisplay<\/code> or pvs<\/code><\/span><\/li>\n
      • To see your combined master pool:<\/strong> vgdisplay<\/code> or vgs<\/code><\/span><\/li>\n
      • To see the striped swap volume we just carved out:<\/strong> lvdisplay<\/code> or lvs\"<\/code><\/span><\/li>\n
        <\/ul>\n
        Take this with a grain of sand I do not know if its true but seems that way.<\/p>\n
        root@g9-pve:~# fdisk -l\r\nDisk \/dev\/sda: 223.57 GiB, 240057409536 bytes, 468862128 sectors\r\nDisk model: KINGSTON SA400S3\r\nUnits: sectors of 1 * 512 = 512 bytes\r\nSector size (logical\/physical): 512 bytes \/ 512 bytes\r\nI\/O size (minimum\/optimal): 512 bytes \/ 512 bytes\r\nDisk \/dev\/sdb: 223.57 GiB, 240057409536 bytes, 468862128 sectors\r\nDisk model: KINGSTON SA400S3\r\nUnits: sectors of 1 * 512 = 512 bytes\r\nSector size (logical\/physical): 512 bytes \/ 512 bytes\r\nI\/O size (minimum\/optimal): 512 bytes \/ 512 bytes\r\nDisk \/dev\/sdc: 223.57 GiB, 240057409536 bytes, 468862128 sectors\r\nDisk model: KINGSTON SA400S3\r\nUnits: sectors of 1 * 512 = 512 bytes\r\nSector size (logical\/physical): 512 bytes \/ 512 bytes\r\nI\/O size (minimum\/optimal): 512 bytes \/ 512 bytes\r\nroot@g9-pve:~# pvs\r\nPV VG Fmt Attr PSize PFree\r\n\/dev\/sda pve-fast lvm2 a-- <223.57g 222.23g\r\n\/dev\/sdb pve-fast lvm2 a-- <223.57g 222.23g\r\n\/dev\/sdc pve-fast lvm2 a-- <223.57g 222.23g\r\n\/dev\/sde3 pve lvm2 a-- <29.22g 3.63g\r\nroot@g9-pve:~# vgs\r\nVG #PV #LV #SN Attr VSize VFree\r\npve 1 2 0 wz--n- <29.22g 3.63g\r\npve-fast 3 1 0 wz--n- <670.70g 666.69g\r\nroot@g9-pve:~# lvs\r\nLV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert\r\ndata pve twi-a-tz-- <10.79g 0.00 1.58\r\nroot pve -wi-ao---- <12.80g\r\nfast-swap pve-fast -wi-ao---- <4.01g<\/pre>\n<\/div>\n
        <\/span>Back to Storage<\/span><\/h2>\n
        So now I just need another logical volume for the VM high speed OS vHDDs.<\/div>\n
        <\/div>\n
        When I tried to assign 100% of the remaining space to the VM data volume, I hit a common LVM roadblock: the thin-pool conversion failed due to insufficient free space (0 extents). This happens because an LVM-Thin pool needs a tiny bit of unallocated space left over to build its hidden metadata index for tracking snapshots. To fix this, I deleted the raw volume and recreated it using 99%FREE of the remaining pool instead. This small tweak left plenty of breathing room for the tracking database while keeping the 3-disk stripe perfectly aligned.<\/div>\n
        <\/div>\n
        \n
        <\/span>1: Recreate it with 99% of the pool space<\/strong><\/span><\/h3>\n
        By allocating 99%FREE<\/code> instead of 100%FREE<\/code>, we guarantee there is plenty of room left over for the metadata engines while still satisfying the stripe alignment requirements.<\/div>\n
        \n
        \n
        \n
        \n
        lvcreate -l <\/span>99<\/span>%FREE -i <\/span>3<\/span> -I <\/span>64<\/span>k -n fast-data pve-fast<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
        \n
        <\/span>2: Convert it to a Thin Pool<\/span><\/strong><\/span><\/h3>\n<\/div>\n<\/div>\n<\/div>\n
        \n
        \n
        \n
        \n
        lvconvert --type thin-pool pve-fast\/fast-data<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
        Once it says successfully converted, run the final step to link it to your Proxmox dashboard:<\/div>\n
        \n
        \n
        \n
        \n
        pvesm add lvmthin Striped-SSDs --vgname pve-fast --thinpool fast-data<\/span><\/code><\/pre>\n
        The conversion went through smoothly, and the high-speed storage tier is now online in the Proxmox GUI under the name Striped-SSDs<\/p>\n
        Quick Sequential I\/O test:<\/p>\n
        root@g9-pve:~# swapoff \/dev\/pve-fast\/fast-swap\r\nroot@g9-pve:~# dd if=\/dev\/zero of=\/dev\/pve-fast\/fast-swap bs=1M count=2000 status=progress conv=fdatasync\r\n2000+0 records in\r\n2000+0 records out\r\n2097152000 bytes (2.1 GB, 2.0 GiB) copied, 3.59334 s, 584 MB\/s\r\nroot@g9-pve:~# swapon \/dev\/pve-fast\/fast-swap\r\nswapon: \/dev\/mapper\/pve--fast-fast--swap: read swap header failed\r\nroot@g9-pve:~# mkswap \/dev\/pve-fast\/fast-swap\r\nSetting up swapspace version 1, size = 4 GiB (4303351808 bytes)\r\nno label, UUID=aa6557d6-626b-4680-9741-2b63a1f55a13\r\nroot@g9-pve:~# swapon \/dev\/pve-fast\/fast-swap<\/pre>\n
        <\/span>More Storage<\/span><\/h2>\n
        Yes even more storage, while we used LVM to stripe across our 3 SSDs. We are going to use Ext4 on the 4TB Drive to host ISOs, or large disk virtual drives on the VMs.<\/p>\n
        <\/span>1. Create a Standard Partition Table<\/strong><\/span><\/h3>\n
        We will write a clean, modern GPT partition table to the raw drive and create a single partition that takes up 100% of the 4TB space.<\/div>\n
        \n
        \n
        \n
        \n
        parted -s \/dev\/sdd mklabel gpt mkpart primary ext4 <\/span>0<\/span>% <\/span>100<\/span>%\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
        \n
        <\/span>2. Format the Partition as Ext4<\/span><\/strong><\/span><\/h3>\n<\/div>\n<\/div>\n<\/div>\n
        Now, we format that fresh partition (\/dev\/sdd1<\/code>) with the standard Linux Ext4 filesystem. This handles sequential data streams beautifully on mechanical platters.<\/div>\n
        \n
        \n
        \n
        \n
        mkfs.ext4 \/dev\/sdd1<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
        <\/div>\n
        <\/span>3. Create a Mount Point and Mount the Drive<\/strong><\/span><\/h3>\n
        We will create a permanent folder on your host OS and mount the physical drive into it.<\/div>\n
        \n
        \n
        \n
        \n
        mkdir -p \/mnt\/pve\/mass-storage\r\nmount \/dev\/sdd1 \/mnt\/pve\/mass-storage\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
        \n
        4. Make the Mount Permanent Across Reboots<\/span><\/strong><\/div>\n<\/div>\n<\/div>\n<\/div>\n
        To make sure Debian hooks this drive back up every time the server boots, we add its unique identification to your filesystem table (fstab<\/code>). Run this command to fetch the drive’s unique ID and automatically write the mount rule:<\/div>\n
        \n
        \n
        \n
        \n
        echo <\/span>\"\/dev\/sdd1 \/mnt\/pve\/mass-storage ext4 defaults,noatime,nofail 0 2\"<\/span> >> \/etc\/fstab\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
        \n
        (Note: noatime<\/code> eliminates unnecessary write cycles to track when files are read, and nofail<\/code> ensures your Proxmox host still boots perfectly even if the 4TB drive is unplugged).<\/em><\/div>\n<\/div>\n<\/div>\n<\/div>\n
        <\/div>\n
        <\/span>5. Register the Directory in the Proxmox GUI<\/strong><\/span><\/h3>\n
        Finally, run this command to tell Proxmox that this folder is ready to accept backup files, ISOs, and VM disks:<\/div>\n
        \n
        \n
        \n
        \n
        pvesm add dir Mass-Storage --path \/mnt\/pve\/mass-storage --content backup,iso,images\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
        \n
        <\/div>\n<\/div>\n<\/div>\n<\/div>\n
        <\/div>\n
        <\/span>Verify Your Entire Server Storage Layout<\/strong><\/span><\/h3>\n
        Now that everything is fully configured, your storage is split perfectly into two distinct, high-efficiency worlds. If you run:<\/div>\n
        \n
        \n
        \n
        \n
        df -h \/mnt\/pve\/mass-storage<\/span><\/code><\/pre>\n
        root@g9-pve:~# parted -s \/dev\/sdd mklabel gpt mkpart primary ext4 0% 100%\r\nroot@g9-pve:~# mkfs.ext4 \/dev\/sdd1\r\nmke2fs 1.47.2 (1-Jan-2025)\r\nCreating filesystem with 976754176 4k blocks and 244195328 inodes\r\nFilesystem UUID: 84d88803-a246-4067-8cec-3a93ba188169\r\nSuperblock backups stored on blocks:\r\n32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,\r\n4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,\r\n102400000, 214990848, 512000000, 550731776, 644972544\r\n\r\nAllocating group tables: done\r\nWriting inode tables: done\r\nCreating journal (262144 blocks): done\r\nWriting superblocks and filesystem accounting information: done\r\n\r\nroot@g9-pve:~# mkdir -p \/mnt\/pve\/mass-storage\r\nroot@g9-pve:~# mount \/dev\/sdd1 \/mnt\/pve\/mass-storage\r\nmount: (hint) your fstab has been modified, but systemd still uses\r\nthe old version; use 'systemctl daemon-reload' to reload.\r\nroot@g9-pve:~# systemctl daemon-reload\r\nroot@g9-pve:~# echo \"\/dev\/sdd1 \/mnt\/pve\/mass-storage ext4 defaults,noatime,nofail 0 2\" >> \/etc\/fstab\r\nroot@g9-pve:~# dd if=\/dev\/zero of=\/mnt\/pve\/mass-storage\/zerofile bs=1M status=progress conv=fdatasync\r\n78979792896 bytes (79 GB, 74 GiB) copied, 293 s, 270 MB\/s\r\nroot@g9-pve:~# dd if=\/dev\/zero of=\/mnt\/pve\/mass-storage\/zerofile bs=1M status=progress oflag=direct\r\n2867855360 bytes (2.9 GB, 2.7 GiB) copied, 33 s, 86.9 MB\/s\r\n\r\n<\/pre>\n
        <\/span>Summary: Proxmox VE Storage Architecture: Maximizing Performance and RAM with a Non-Cached Controller<\/span><\/h2>\n
        When designing local storage for a hypervisor host, the default answer is often to throw everything into a ZFS pool. However, storage architecture should never be a one-size-fits-all solution. For this Proxmox VE build\u2014featuring 50 GB of physical RAM, an HPE B140i controller running in SATA AHCI pass-through mode, a trio of SSDs, and a single 4TB mechanical spindle drive\u2014we chose a hybrid, LVM-centric approach designed specifically to prioritize raw performance and maximize available system memory for virtual workloads.<\/div>\n
        <\/span>Phase 1: Protecting the Boot Media and Reclaiming Swap<\/span><\/h3>\n
        The server boots Proxmox from an internal 32 GB SD card. By default, the Debian-based installer creates an active swap volume directly on the boot drive. Because SD cards utilize low-endurance flash memory, leaving an active swap partition on this media is a hardware hazard that would quickly wear out the card. Furthermore, with 50 GB of physical RAM available, host-level swapping should be incredibly rare.<\/div>\n
        We immediately disabled and purged the default pve-swap<\/code> volume from the SD card. To preserve a host safety net without introducing latency or deadlocks, we moved the swap partition to our incoming solid-state pool. Crucially, this swap space was created as a fixed, pre-allocated “Thick” LVM volume rather than being nested inside a thin data pool, ensuring kernel stability under unexpected memory spikes.<\/div>\n
        <\/span>Phase 2: The Performance Tier (3-Disk LVM-Thin Stripe)<\/span><\/h3>\n
        To give our VM operating systems maximum IOPS and unthrottled throughput, we grouped our three zeroed SSDs (sda<\/code>, sdb<\/code>, sdc<\/code>) into a single LVM Volume Group named pve-fast<\/code>.<\/div>\n
        Because data safety is managed via a strict backup strategy rather than local fault tolerance, we chose to strip data evenly across all three disks using LVM’s interleaved striping parameter (-i 3<\/code>). This acts as a high-efficiency software RAID0 array directly inside the Linux kernel. During configuration, we encountered a classic LVM hurdle: allocating 100% of the remaining pool to a raw data container left 0 extents<\/code> behind for the metadata tracker, causing the LVM-Thin conversion to fail. Re-provisioning the container at 99%FREE<\/code> provided the required breathing room for the tracking database while maintaining perfect alignment across the three controllers.<\/div>\n
        This performance tier consumes virtually 0 MB of host RAM, leaving almost all 50 GB available for our VMs. A sequential write test using dd<\/code> directly against the raw striped blocks clocked in at a blistering 584 MB\/s<\/strong>, successfully compounding the bandwidth of our independent controllers.<\/div>\n
        <\/span>Phase 3: The Mass Storage Tier (Zero-RAM Spindle Directory)<\/span><\/h3>\n
        For our 4TB mechanical drive (sdd<\/code>), we chose to completely bypass LVM and ZFS, formatting it directly as a standard Ext4 Directory partition mapped straight to the Proxmox dashboard.<\/div>\n
        Using ZFS here would have starved our host by demanding a massive chunk of RAM for its ARC cache, while its Copy-on-Write architecture would have severely choked write performance on a controller lacking a battery-backed hardware cache. LVM-Thin was also discarded for this drive; Proxmox backup files require a standard filesystem folder, and thin block-level provisioning creates massive physical fragmentation on spinning platters over time.<\/div>\n
        By using a standard Ext4 directory, we can provision large virtual hard disks (vHDDs) for backup servers like Veeam using the QCOW2<\/strong> file format. QCOW2 handles thin-provisioning intelligently at the virtual file level, preventing the hypervisor from scattering blocks chaotically across the physical disk.<\/div>\n
        <\/span>Phase 4: Benchmarking and the Reality of Caching<\/span><\/h3>\n
        We ran two distinct write benchmarks against our newly formatted 4TB Ext4 storage directory to observe how the operating system handles a mechanical drive:<\/div>\n
          \n
        1. The Buffered Test (conv=fdatasync<\/code>):<\/strong> After an initial RAM-buffered burst, the sequential write stream stabilized at an impressive 270 MB\/s<\/strong>. This is significantly faster than the drive’s raw hardware capability. The boost is entirely driven by Ext4 optimizations like Delayed Allocation (delalloc<\/code>) and sequential extents, which neatly arrange incoming data blocks on the fast, outer edge of the empty platter.<\/span><\/li>\n
        2. The Direct I\/O Test (oflag=direct<\/code>):<\/strong> To expose the raw physical limits of the drive, we bypassed the Linux kernel’s RAM page cache completely. Stripped of its file-system optimizations, the performance leveled off at 100 MB\/s<\/strong>, exposing the exact mechanical floor of the spindle and proving how vital the filesystem’s caching layer is for normal operation.<\/span><\/li>\n<\/ol>\n
          <\/span>The Power-Safety Tradeoff<\/span><\/h3>\n
          The 270 MB\/s buffered speed comes with an engineering tradeoff: write safety. Because our AHCI pass-through controller lacks a physical battery-backed write cache, any data floating in the host’s volatile RAM cache during a sudden power outage will be lost.<\/div>\n
          While this risk would be unacceptable for a live production database, it is perfectly suited for this specific architecture. The 4TB tier is dedicated strictly to static ISOs and compressed Veeam backup repositories; a power failure simply invalidates a running backup job, which can easily be restarted once the system boots back up. To completely mitigate this, the host will be plugged into an Uninterruptible Power Supply (UPS) integrated with automated shutdown software, ensuring all memory caches are safely flushed to the physical disks before the server powers down.<\/div>\n
          This finished architecture leaves us with a highly optimized, dual-tier environment: a blazing fast 584 MB\/s SSD stripe<\/strong> for active VMs, a highly efficient 270 MB\/s sequential mass storage folder<\/strong> for backups, and a completely unburdened 50 GB pool of RAM dedicated entirely to running workloads.<\/div>\n<\/div>\n<\/div>\n
          <\/div>\n
          This is the bare basics of setting up a Proxmox server. Things we haven’t covered yet are networking, updating, clustering, shared storage, managing VMs, etc. These will be covered in the upcoming blog posts. This one is just the fundamental requirement to all those other topics. This is just the foundation. Hope this helps someone.<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
          Install Proxmox Hardware Step 1) Pick Hardware. Important is CPU support. Mostly ARM or x86_64. – My host a HPE DL160 G9. Software Step 2) Install using appropriate installer image. – In my case x86_64 version 9.2 – I installed to an internal 32GB sd card. Storage (Physical) Step 3) VM Storage. I’ve discussed this … <\/p>\n
          Continue reading “Setting up a Proxmox host”<\/span><\/a><\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"sfsi_plus_gutenberg_text_before_share":"","sfsi_plus_gutenberg_show_text_before_share":"","sfsi_plus_gutenberg_icon_type":"","sfsi_plus_gutenberg_icon_alignemt":"","sfsi_plus_gutenburg_max_per_row":"","footnotes":""},"categories":[5,8],"tags":[460,456,461,501],"class_list":["post-1823","post","type-post","status-publish","format-standard","hentry","category-hypervisors","category-server-administration","tag-lvm","tag-proxmox","tag-pve","tag-storage"],"_links":{"self":[{"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/posts\/1823","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/comments?post=1823"}],"version-history":[{"count":1,"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/posts\/1823\/revisions"}],"predecessor-version":[{"id":1824,"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/posts\/1823\/revisions\/1824"}],"wp:attachment":[{"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/media?parent=1823"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/categories?post=1823"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/tags?post=1823"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}