How big should the SSDs in a ZFS special vdev be, and should you add one at all?

1. 1.Check the pool is the kind a special vdev is meant for. The Proxmox guide says a special device can improve the speed of a pool of slow spinning disks with a lot of metadata changes, and replies in one Level1Techs thread said the benefit is for spinning-disk pools. By default the special class takes all pool metadata, the indirect blocks of user data and any deduplication tables; small file and zvol blocks go there only if you opt in.
2. 2.List what each dataset would send there today. special_small_blocks is the opt-in threshold for small data blocks; recordsize is the dataset's block size, and the two are read together.

   zfs get -r special_small_blocks,recordsize <pool>

   Readingspecial_small_blocks=0, the default, means only metadata would go to the special vdev. A value at or above a dataset's recordsize sends all of that dataset's data there. The SOURCE column shows whether a value is inherited from the pool's root dataset.
3. 3.Read the pool's block statistics. zdb reads the pool devices directly from userspace; its man page warns that on an imported, active pool it may, though unlikely, interpret inconsistent pool data and behave erratically, so run it when the pool is quiet. -L skips leak detection and space map loading. TrueNAS forum posts add -U /data/zfs/zpool.cache, the zdb(8) option for a cache file other than /etc/zfs/zpool.cache.

   zdb -Lbbbs <pool>

   ReadingFrom -bb upwards zdb prints a per-type table (Blocks, LSIZE, PSIZE, ASIZE, avg, comp, %Total, Type) and a 'Block Size Histogram' whose psize, lsize and asize columns each give Count, Size and a cumulative Cum. total. The histogram comes from zdb.c; the zdb(8) man page we read does not describe it. The community sizing guide on the TrueNAS forums takes the metadata figure from the per-type table's L1 Total ASIZE and reads the asize Cum. value on the row for a candidate special_small_blocks size as the data that cutoff would send to the special class. That is a community method, not an OpenZFS formula.
4. 4.Turn the two figures into a capacity. Add the metadata figure to the small-block figure for the cutoff you intend to set, or use the metadata figure alone if special_small_blocks will stay at 0. Then allow for the reserve in zfs(4): with zfs_special_class_metadata_reserve_pct at its default of 25, small data blocks are placed on the special and dedup vdevs only while their free space exceeds 25%, so once metadata and small blocks together fill three quarters of the mirror, further small blocks go to the hard drives.
   ReadingWe found no OpenZFS, Proxmox or TrueNAS document giving a sizing rule as a percentage of the pool or GB per TB, which is why this page measures rather than quoting one. A zdb estimate taken on blocks of one size can miss on blocks of another: in one Level1Techs thread zdb on the old pool, which used the default recordsize, suggested about 100GB, and after an rsync into the new pool, which was created with recordsize=1M, the special vdev held about 1.3GB. The histogram was measured on a different block-size population, and the thread did not resolve the discrepancy (see the third path).

1. 1.Check how full the special class is.

   zpool list -v <pool>

   ReadingPer-vdev rows with SIZE, ALLOC, FREE, FRAG and CAP. The zpool-list(8) example we read shows only a raidz1 pool; that the special mirror appears as its own row group comes from forum output. Its ALLOC and CAP are how full the special class is. By default small data blocks stop going there once its free space is 25% or less, and when it is full new allocations spill to the data vdevs. In one TrueNAS forum thread an 894G mirror beside 34.8T allocated on HDDs was 73.1% full.
2. 2.Get the same figure from zdb: run the block statistics from the first path and find the 'Special class' line, which zdb prints when a special class exists.
   ReadingThe line gives the bytes allocated in the special class and 'used: N%'. A 'Dedup class' line follows when the pool has a dedup class.
3. 3.If it is fuller than metadata explains, compare special_small_blocks with recordsize for each dataset, using the zfs get command from the first path. The Proxmox guide warns that a value greater than or equal to the dataset's recordsize writes all of that dataset's data to the special device. Placement follows the size of each block as written, not the dataset's current recordsize: blocks received by zfs send/receive keep the sizes they had on the source, so a threshold below the current recordsize can still catch them. A reply in the TrueNAS thread with the 73% full vdev also pointed out that the tail blocks of large files can qualify as small blocks.
   ReadingIn one Level1Techs thread the new pool was created with recordsize=1M and the receive used special_small_blocks=512K, below that recordsize, yet a zfs send/receive put almost all of the data on the special vdev. Replies said send/receive keeps the source's block sizes, and the source pool used the default recordsize.
4. 4.To stop a dataset sending data blocks there, lower its threshold or set it to 0. The two documents disagree on the boundary and the range, so we print both: OpenZFS's zfsprops(7) says blocks smaller than or equal to the value, after compression and encryption, go to the special class, with values up to 16 MiB on the current man page; the Proxmox guide says blocks smaller than the value, with a power of two from 512B to 1M.

   zfs set special_small_blocks=0 <pool>/<dataset>

   ReadingProxmox's wording covers new file blocks. We found no OpenZFS, Proxmox or TrueNAS document stating what happens to blocks already on the special vdev when the value changes.

1. 1.Start from what the documents say. The Proxmox guide says that after the property is set, new file blocks smaller than the size are allocated on the special device. We found no OpenZFS, Proxmox or TrueNAS document stating that blocks already written stay where they are when special_small_blocks changes, or that they move. The 'new file blocks' wording is the closest any of them comes.
2. 2.Do not count on zfs rewrite to move data onto the special vdev. The zfs-rewrite(8) page we read says it rewrites blocks as is at a new location, possibly with new properties, and that changes to properties affecting logical block size, such as recordsize, have no effect. It does not mention special_small_blocks or the special class. In one Level1Techs thread, an owner set special_small_blocks=64K and ran zfs rewrite -rv, and the special vdev reached about 23G against an estimated 75G; a reply said new vdevs generally receive only new writes, and the thread ended unresolved.
3. 3.Know how the copy method changed the outcome in one forum case. On a raidz1 pool with recordsize 1M, an rsync at special_small_blocks=64K put about 1.3GB on the special vdev; a zfs send/receive at 512K then put almost all of the data there. Replies explained that send/receive keeps the source's block sizes while rsync re-blocks the files, and an rsync into new datasets at 512K worked for the owner. That is one thread's experience, not a documented migration method.

1. 1.Set the redundancy first. zpoolconcepts(7) says the special device's redundancy should match or exceed that of the pool's normal devices. The Proxmox guide calls it a point of failure for the whole pool, and TrueNAS says that if it fails with no redundancy the pool is corrupted and its data cannot be reached. TrueNAS's development documentation for TrueNAS 28 gives the pairing: a RAIDZ2 data vdev takes a RAIDZ2 or three-way-mirror metadata vdev, and pool creation fails when the metadata vdev is less redundant. At the command line, zpool add has -f and --allow-replication-mismatch, which accept a conflicting replication level; neither changes what a failed special vdev costs.
2. 2.Check whether you could take it out again, because the documents disagree. OpenZFS's zpool-remove(8) removes a special vdev only when the pool's primary storage has no raidz or draid top-level vdev, every top-level vdev shares one ashift, encryption keys are loaded and the device_removal feature is enabled. TrueNAS 25.04 says a metadata vdev on a RAIDZ pool is a permanent addition, removable only when the data vdevs are mirrors. The Proxmox guide says, with no conditions, 'Adding a special device to a pool cannot be undone'. zpool-add(8) adds that top-level vdevs with different sector sizes prohibit future removal.
   ReadingOn a raidz pool all three agree it stays; zpool-remove(8) also rules out draid. Two forum sources also report that a special vdev could not be replaced with smaller devices, one quoting the error 'device is too small'.
3. 3.Do not plan on a hot spare for it without checking your TrueNAS version. The 25.04 documentation suggests mirroring metadata vdevs and optionally adding hot spares; the TrueNAS 28 development documentation says hot spares only activate in data vdevs, so a hot spare cannot replace a failed drive in a special vdev.
4. 4.Pick the SSDs. The Proxmox guide says to use fast SSDs for the special device. TrueNAS 28 documentation says that when the SSDs have an internal cache, a UPS helps minimise the risk from power loss. OpenZFS defines adequate power-loss protection as protecting the drive's own metadata and all flushed data, and says hardware power-loss protection is a feature of enterprise SSDs. We found no OpenZFS, Proxmox or TrueNAS document stating an endurance (DWPD or TBW) or power-loss protection requirement specifically for a special vdev.
5. 5.Add the mirror. This is irreversible on any pool with raidz or draid data vdevs, and from then on the special vdev is a point of failure for the whole pool: lose it without redundancy and you lose the pool and its data. The pool must already have at least one normal data vdev. Name three devices after 'mirror' where the data vdevs are RAIDZ2, per the TrueNAS pairing above.
   Destructive. This command destroys what is on the device and cannot be undone. Confirm the device path twice and have a backup of anything you still need before running it.

   zpool add <pool> special mirror <dev1> <dev2>

   ReadingThis is the form the Proxmox guide documents. In the one forum report in the third path where a special vdev was added to a pool already holding data (t5), a reply said new vdevs generally receive only new writes, and even after zfs rewrite the vdev held about 23G against an estimated 75G. New metadata, and new small blocks once you opt in, go to it.
6. 6.If your measurement supports a small-block cutoff, set it per dataset. zfsprops(7) says a special class vdev must be added to the pool before this property is set. Keep the value below the dataset's recordsize unless you mean to put the whole dataset on the SSDs.

   zfs set special_small_blocks=<size> <pool>/<dataset>

   ReadingIn a TrueNAS forum thread about photo and video editing on two 4-wide RAIDZ1 vdevs, replies suggested recordsize=1M with special_small_blocks=512K and a three-way special mirror for RAIDZ2; no outcome was posted.

Not from any OpenZFS, Proxmox or TrueNAS document we read. Forum figures are anecdotes and spread widely: in one ServeTheHome thread replies offered about 0.3% of the pool for metadata, 3-5%, and about 10% with small blocks, with no formula agreed; in another, a 121T pool 47% full had 38.7G on a three-way mirror of about 932G SSDs, 38.7G being 4.17% of the special mirror's own capacity; on Level1Techs one reply sized from its own histogram at about 35GB of sub-1M blocks on a 42.7TB pool. Measure your own pool with zdb instead.

zpoolconcepts(7) lists the intent log among what the special class holds when the pool has no separate log device. zfs(4) has zil_special_is_slog, default 1: when it is enabled and written blocks go to normal vdevs, present special vdevs are treated as SLOGs, and the parameter is ignored if a SLOG is present. Its sync-write traffic is therefore part of what the SSDs see on a pool without a log device.

We found no OpenZFS, Proxmox or TrueNAS document covering replacement with smaller drives. Two forum sources report that it fails: a Level1Techs poster got 'device is too small', and the TrueNAS community sizing guide says a special vdev cannot be replaced with smaller drives. Whether the vdev can be removed instead depends on the zpool-remove(8) conditions, which rule out any pool with raidz or draid data vdevs.

Deduplication tables go to the special class by default: zpoolconcepts(7) lists them, and zfs(4) has zfs_ddt_data_is_special, default 1. A pool using dedup therefore puts its tables on the special vdev too, unless the pool has a separate dedup vdev, and zfs(4) applies the same 25% small-block reserve to dedup vdevs as to special ones.