TL;DR: zfs set sync=disabled your/proxmox/dataset
Lately I’ve been playing around with Proxmox installed on an Intel NUC (the cleverly named NUC6CAYH, to be precise), and I must say it is really, really, cool.
I usually store my containers and VMs on the local 180 GB SSD that used to be in my old MacBook Pro, since it’s reasonably fast and it works well, but I wanted to experiment with NFS-backed storage off my FreeNAS box (4x4TB WD Reds in RAIDZ1, 16 GBs of RAM, an i5–3330 processor).
Frankly, I was pretty unsatisfied with the performance I was getting. Everything felt pretty slow, especially compared to the internal SSD and, surprisingly, to storing the same data on a little WD MyCloud (yes, the one with the handy built-in backdoor).
My very unscientific test was creating a fresh container based on Ubuntu 16.04, and upgrading the stock packages that came with it. As of today, it meant installing around 95 MB’s worth of packages, and a fair bit of I/O to get everything installed.
The task was completed in around 1’30″ with the container on the internal SSD, 2’10″ on the WD MyCloud, and an embarassing 7’15″ on the FreeNAS box.
After a bit of googling, I came to an easy solution: set the sync property of the ZFS dataset used by Proxmox to disabled (it is set to standard by default).
The complete command is zfs set sync=disabled your/proxmox/dataset (run that on FreeNAS as root or using sudo).
To be honest, I don’t really know the data-integrity implications of this flag: both machines and the switch between them are protected from power failures by two UPSs, so that shouldn’t be much of an issue.
Anyway, just changing that little flag signlificantly reduced the time required to complete my “benchmark”, bringing it down to around 1’40″, very close to Proxmox’s internal SSD. Again, at the moment I don’t really need to run VMs/CTs off the FreeNAS storage, but it is good to know that it is possible to achieve much faster performance with this little tweak.
2 replies on “Improve FreeNAS NFS performance when used with Proxmox”
Thank you for the article.
I’m currently presenting lots of problems using FreeNAS as my storage backend (NFS or iSCSI). Even taking the data out of the dataset to a USB3.0 SSD its super slow.
I will try disabling sync like you said and see what happens.
Thanks again!
Regards,
Alvaro Lamadrid
sync=disabled basically tells FreeNAS to do unprotected write caching. In short, when a write request comes in from a host or VM, FreeNAS with sync=disabled will write it to RAM and acknowledge write success back to the host or VM immediately without committing it to disk or some kind of permanent storage. Most of the time, this is safe for home labs with a UPS, but not recommended for business or datacenters.
Imagine you home loses power and your UPS runs out of juice. You’ll most likely get data corruption because of the uncommitted write transactions.