{"id":1871,"date":"2026-09-27T16:32:55","date_gmt":"2026-09-27T21:32:55","guid":{"rendered":"https:\/\/zewwy.ca\/?p=1871"},"modified":"2026-09-27T16:32:55","modified_gmt":"2026-09-27T21:32:55","slug":"pve-a-storage-deep-dive-lvmthin-vs-zfs-vs-ext4","status":"publish","type":"post","link":"https:\/\/zewwy.ca\/index.php\/2026\/09\/27\/pve-a-storage-deep-dive-lvmthin-vs-zfs-vs-ext4\/","title":{"rendered":"PVE: A Storage Deep Dive – LVMThin vs ZFS vs EXT4"},"content":{"rendered":"

<\/span>Testing different Storage Architectures against a cheap consumer SSD<\/span><\/span><\/h1>\n
When I first spun up my virtualization host, I knew that storage configuration would make or break my performance. But nothing prepared me for just how wildly my benchmarks would swing based purely on software definitions. Using the exact same underlying physical SSD, I ran a series of tests to uncover how different virtual controllers, file systems, and storage architectures handle real-world disk operations.<\/div>\n
Here is the raw data from my benchmarking journey, followed by a breakdown of what I learned, and a definitive pros and cons guide for every setup.<\/div>\n
<\/div>\n
<\/div>\n

<\/span>\ud83d\udcca The Cumulative Benchmarks: My Raw Results<\/span><\/h2>\n
To keep the comparisons fair, I tracked sequential and random operations across four distinct iterations of my storage stack.<\/div>\n
<\/div>\n
<\/div>\n
1. The Baseline: LVMthin with a Virtual SATA Controller (64 MiB Test)<\/div>\n
My journey started here, and the results were a painful wake-up call to the limits of legacy hardware emulation.<\/div>\n
<\/div>\n
\n
\n
\n
\n
<\/div>\n<\/div>\n
\n
[Read]  SEQ Q8T1:  429.221 MB\/s  |  SEQ Q1T1:  318.640 MB\/s  |  RND Q32T1:  13.760 MB\/s  |  RND Q1T1:   5.507 MB\/s\r\n[Write] SEQ Q8T1:   44.661 MB\/s  |  SEQ Q1T1:   16.980 MB\/s  |  RND Q32T1:   2.382 MB\/s  |  RND Q1T1:   1.655 MB\/s\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
\n
2. The Protocol Flip: LVMthin upgraded to VirtIO SCSI Single (64 MiB Test)<\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n
By changing a single dropdown menu in my VM settings from SATA to VirtIO SCSI, I unbottlenecked the communication protocol.<\/div>\n
\n
\n
\n
\n
[Read]  SEQ Q8T1:  493.329 MB\/s  |  SEQ Q1T1:  383.567 MB\/s  |  RND Q32T1:  88.362 MB\/s  |  RND Q1T1:  12.480 MB\/s\r\n[Write] SEQ Q8T1:   90.589 MB\/s  |  SEQ Q1T1:   21.391 MB\/s  |  RND Q32T1:   2.403 MB\/s  |  RND Q1T1:   1.879 MB\/s\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
\n
3. The RAM Illusion: ZFS with VirtIO SCSI Single (Cached 64 MiB Test)<\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n
Next, I swapped my backend volume manager to ZFS. With a tiny test file size, I accidentally benchmarked the sheer speed of my host’s RAM via the ZFS ARC cache.<\/div>\n
\n
\n
\n
\n
[Read]  SEQ Q8T1: 4401.052 MB\/s  |  SEQ Q1T1: 1921.489 MB\/s  |  RND Q32T1:  84.354 MB\/s  |  RND Q1T1:  19.714 MB\/s\r\n[Write] SEQ Q8T1:  205.138 MB\/s  |  SEQ Q1T1: 1791.981 MB\/s  |  RND Q32T1:  68.788 MB\/s  |  RND Q1T1:  17.412 MB\/s<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n<\/div>\n
<\/div>\n
<\/div>\n
4. The True Hardware Limit: ZFS with VirtIO SCSI Single (Raw 8 GiB Test)<\/div>\n
By cranking the test file size up to 8 GiB, I forced the data straight through the RAM buffers and onto the raw NAND flash chips. This represents my real physical drive performance under ZFS.<\/div>\n
\n
\n
\n
\n
[Read]  SEQ Q8T1:  739.249 MB\/s  |  SEQ Q1T1:  715.466 MB\/s  |  RND Q32T1:  77.281 MB\/s  |  RND Q1T1:  17.737 MB\/s\r\n[Write] SEQ Q8T1:  382.726 MB\/s  |  SEQ Q1T1:  219.378 MB\/s  |  RND Q32T1:  49.969 MB\/s  |  RND Q1T1:  15.344 MB\/s\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
\n
5. The Image File Contender: qcow2 on an EXT4 Host File System (64 MiB Test)<\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n
Finally, I tested the classic approach\u2014placing a qcow2 virtual disk file on top of a standard Linux EXT4 partition.<\/div>\n
\n
\n
\n
\n
[Read]  SEQ Q8T1:  189.151 MB\/s  |  SEQ Q1T1:  154.374 MB\/s  |  RND Q32T1:  41.495 MB\/s  |  RND Q1T1:  10.420 MB\/s\r\n[Write] SEQ Q8T1:  124.732 MB\/s  |  SEQ Q1T1:  115.363 MB\/s  |  RND Q32T1:   2.419 MB\/s  |  RND Q1T1:   1.726 MB\/s\r\n<\/span><\/code><\/pre>\n<\/div>\n<\/div>\n
\n
<\/div>\n<\/div>\n<\/div>\n<\/div>\n
<\/div>\n
<\/span>\ud83d\udd0d Structural Deep Dive: Pros, Cons, and My Key Architectural Takeaways<\/span><\/h2>\n
Every storage backend has a specific target audience. Based on my benchmarking data and the structural architecture of these systems, here is how they stack up.<\/div>\n
<\/div>\n
<\/div>\n
<\/span>Setup A: LVMthin (Block Storage)<\/span><\/h3>\n
LVMthin acts as a pure logical volume manager natively inside the Linux kernel. It gives the VM direct block-level access to the storage layer without an intervening file system.<\/div>\n
    \n
  • The VirtIO SCSI Upgrade Difference:<\/strong> My data showed a 542% explosion<\/strong> in random reads (from 13.76 MB\/s to 88.36 MB\/s) just by moving away from the virtual SATA controller. SATA forced my VM into a shallow, serialized hardware queue. Upgrading to VirtIO SCSI unlocked deep parallel queue depths (Q=32), passing commands straight down to the kernel block layer efficiently.<\/span><\/li>\n
    <\/ul>\n
    <\/div>\n
    Pros<\/div>\n
      \n
    • Ultra-Low Overhead:<\/strong> It consumes almost zero host RAM and negligible CPU power.<\/span><\/li>\n
    • Zero Write Amplification:<\/strong> Blocks are written directly to the SSD, maximizing the physical lifespan (TBW) of consumer drives.<\/span><\/li>\n
      <\/ul>\n
      <\/div>\n
      Cons<\/div>\n
        \n
      • No Cache Cushioning:<\/strong> Because it lacks an aggressive RAM cache layer, your performance is entirely capped by the physical drive hardware. My random writes hit a miserable 2.4 MB\/s wall<\/strong> in both tests because the raw SSD couldn’t keep up with heavy random write thrashing.<\/span><\/li>\n
      • Feature Light:<\/strong> It lacks advanced data integrity features like inline file self-healing.<\/span><\/li>\n
        <\/ul>\n
        <\/div>\n
        \n
        <\/div>\n
        <\/span>Setup B: ZFS (Advanced Copy-on-Write File System)<\/span><\/h3>\n
        ZFS merges the file system and volume manager into a single entity. It uses a Copy-on-Write (CoW) design, meaning existing data is never overwritten; modifications are written to entirely new blocks, and metadata addresses are updated afterward.<\/div>\n
          \n
        • The RAM vs. Real Disk Difference:<\/strong> In my 64 KiB test, ZFS hid my operations completely inside the Adaptive Replacement Cache (ARC)<\/strong> and transaction memory, showing an insane 4.4 GB\/s read rate<\/strong>. Once I used an 8 GiB file to smash past the RAM barrier, I saw my true physical drive speeds (~740 MB\/s Read \/ ~380 MB\/s Write<\/strong>).<\/span><\/li>\n
        • The Random Write Fix:<\/strong> Remarkably, ZFS managed to boost my raw un-cached random writes up to 49.97 MB\/s<\/strong> (compared to LVMthin’s 2.4 MB\/s). This is because ZFS groups messy random writes together in memory and flushes them to the physical disk as a clean, orderly sequential stream.<\/span><\/li>\n
          <\/ul>\n
          <\/div>\n
          Pros<\/div>\n