Crazy ZFS ZIL options: md(4) umass(4)

[Daniel Kalchev]

Some further clarification.

ZIL is only read on crash. If you shut down properly etc, the ZIL will be never read. For most of it's lifetime, the ZIL is write-only safety net. An insurance, if you wish.

Because it is write-only, cheap USB FLASH devices are the poorest candidates. USB FLASH devices, even the most "high performance" are mediocre at best. Because SLOG (separate ZIL) is written sequentially, even the slowest spinning disk has higher throughput.  You want "write-optimised" device for the SLOG, which no USB FLASH device is, ever. By the way, you may have better luck with professional photography write-optimised CF cards, that are capable of recording large amounts of huge raw images in a burst -- but these typically are more expensive than an write-optimised SSD and you don't really need much capacity. But again: forget about USB if you don't enjoy problems. :)

By separating the ZIL, you avoid fragmentation, yes. This is because in-pool ZIL uses variable length records, that are then removed, leaving variable length holes. ZFS tries to work around this, but it's never perfect.

If you use separate mirrored disks for the ZIL, you will 

- keep writing sequential (no seeks from interfering reads);
- have a reliable recovery medium for the ZIL as long as at least one of the drives survives the outage.

If you want ultimate performance, the solution is battery backed RAM drives for ZIL.

Daniel