In a few minutes, browse the home lab forums and you’ll come across dozens of tips for maximizing the capabilities of your Network Attached Storage server. Many of these as activation SMB multichannel and using an SSD as a boot drive actually makes a lot of sense and can improve the performance of your storage server. But you’re likely to encounter random service startup tips — some of which may do more harm than good.
SSD caches are one such aspect because there is more to them than meets the eye. If you use ZFS As a filesystem major like I do, you’ll often see posts about the three types of caches (well, technically two, because calling the last one a cache would be a bit inaccurate) and how they can speed up your NAS tasks. But after getting acquainted with each of them, I have to admit that they are not worth it for the average consumer. At best, they can really improve niche tasks, while more complex setups can wreak havoc on your hard drive.
L2ARC speed only works under special conditions
Not very useful for home servers
Starting with the most common SSD cache that people recommend using, the read-only Level 2 Adaptive Swap Cache is designed to complement RAM-based ARC. Let’s say you have 16GB of storage on your NAS, 50% of which is assigned to the RAM cache by TrueNAS. Depending on the storage capacity of the NAS, 8GB ARC may be a bit too low, so you can assign hundreds of GB from any old SSD as secondary cache in the form of L2ARC. On paper, the extra capacity should significantly increase transfer speeds, along with SSD’s faster speeds (compared to HDDs).
Unfortunately, there are a few annoying quirks with SSD-powered read cache. For starters, constantly writing to the cache can reduce the lifespan of an SSD. So, if you don’t want your expensive NVMe drive to be exposed to frequent caching operations, you may prefer high-endurance drives. Then there’s the fact that L2ARC only boosts read speeds for data you access frequently, meaning everything from backups to casual movie sessions won’t really benefit from SSD cache. Basically, you’d be wasting the SSD’s endurance without appreciably improving your home lab tasks.
Of course, you should experience higher speeds when accessing the same set of files via your L2ARC setup. But unless you’re running VMs from a NAS, writing databases all the time, or running games from network drives (yes, I’ve tried that and it works surprisingly well), configuring a read cache on an SSD doesn’t make much sense. L2ARC can even speed up read operations in the first place. On a system that doesn’t need RAM, you can spend valuable RAM space to accommodate the L2ARC mapping table, thereby causing additional latency. Combine all this with random caching and you can see why L2ARC is not worth using in a home lab environment.
SLOG only enforces synchronous write operations
And even then, it’s designed to add more continuity to your file transfers
Separate Intent Registration can technically improve performance for write tasks, but it only affects those that are synchronous in nature. For reference, most tasks on a typical NAS are asynchronous, meaning that the write task is considered complete as soon as it is written to RAM, instead of being stored on the HDD as you might expect. The problem with asynchronous tasks is that if the power goes out before the file system transfers the data from RAM to the hard disk, the entire operation will fail. Files involved in synchronous write operations, on the other hand, are first sent to RAM and written to a persistent data unit (what you’d typically call a ZFS Intent Record) until the operation is considered incomplete.
Normal HDD-only pool sync operations may take a while to complete, as the file will not be considered complete until it has been written to the very slow ZIL on your hard drive(s). Throwing an SSD as a ZIL can speed up synchronous tasks quite a bit, especially if you store all VMs or databases on a network share on the NAS. However, any backup operations and media archive tasks will not benefit from your SLOG storage. Given that you don’t need excruciatingly slow sync operations for typical NAS workloads anyway, SLOG is just another unnecessary addition to a mid-range storage hub.
Heck, it’s not even a cache to begin with
Every SSD cache I’ve mentioned so far has one thing in common: losing it won’t affect the actual storage pool in which it stores its data. If the drive carrying the L2ARC read cache fails, you can simply replace it and continue your backup operations as usual. For SLOG drives, it’s best to opt for a mirrored setup – not because losing it will break your array, but because a wrong SLOG configuration will cause your sync operations to revert to the slower HDD-based ZIL. Sure, if the SLOG drive fails in the middle of a transfer, you might lose files you want to keep on your NAS drives, but the overall pool shouldn’t be compromised.
But if you go down the Custom Metadata VDEV route, the situation is radically different. Heck, it’s not even a cache, but seeing as how many people view it as one, I have to get its shortcomings off my chest. If you configure the SSD as a Dedicated Metadata VDEV, it will load smaller chunks of data (including metadata) from the HDD to the blazing fast drive. Interestingly, this feature will really improve performance when you access folders in your pools, perform indexing operations on archived media, and perform scrubbing tasks.
warning? A dedicated VDEV is part of your storage pool, and if something happens to that SSD, you’ll lose all the data stored in the shared pool. So to ensure that the Custom VDEV doesn’t ascend to tech heaven and take the rest of your files with it, you’ll need to invest in lots of high-endurance drives and configure them in a mirrored setup. Given the prices of high-end consumer SSDs, let alone their enterprise-class counterparts, and considering that NAS units with enough drive bays to support unnecessary Custom VDEV installations for all of your storage pools can cost an arm and a leg, I can’t recommend enabling this feature on your storage server.
But you can still use old SSDs in a NAS just fine
To be brutally honest with you, I started researching SSD storage to stop the high-speed NVMe and SATA drives I pulled from old hardware from gathering dust. But here’s the thing: throwing them on a NAS as storage pools for frequently accessed data makes more sense than configuring L2ARC or SLOG setups for typical home labs. As long as you have decent Ethernet speeds, you can simply enable network sharing on your old SSD and use it to transfer files at high speeds. This is my experience with running Steam games from NAS created and I still have some storage headers stored on an iSCSI share mounted on my PCIe Gen 3 drive. Likewise, it can serve as a neat boot pool for experimental virtual guests, although if you’re trying to do anything even remotely important with old SSD-stored VMs and containers, you might want to move them to the proper queue.
