Call for testers with NVMe

On 2026-09-20 01:31, Abdelkader Boudih wrote:

> Your HP EX900+ is a budget drive. The controller was never built to reach that speed. Having access to the Autobahn in Germany doesn't suddenly make your car go 300 km/h. :)

Oh damn, but that would be really awesome. ;)

My understanding is that "nvmecontrol perftest" performs only interface
throughput test and it should be close to theoretical max, while "dd
iflag=direct" does real sequential data reads bypassing local cache.

> Try: 
> 
> nvmecontrol perftest -n 32 -o read -s 4096 -t 30 nvme0ns1 
> nvmecontrol perftest -n 32 -o read -s 131072 -t 30 nvme0ns1 
> 
> and maybe try with -n 64. 
> 
> I would be interested in seeing how the IOPS and bandwidth scale with the number of outstanding requests in those drives.

  Here's what I get:

root@megamini:/home/rz # nvmecontrol perftest -n 16 -o read -s 4096 -t
30 nvme0ns1
Threads: 16 Size: 4096 READ Time: 30 IO/s: 106826 MB/s: 417

root@megamini:/home/rz # nvmecontrol perftest -n 16 -o read -s 131072 -t
30 nvme0ns1
Threads: 16 Size: 131072 READ Time: 30 IO/s: 16991 MB/s: 2123

root@megamini:/home/rz # nvmecontrol perftest -n 32 -o read -s 4096 -t
30 nvme0ns1
Threads: 32 Size: 4096 READ Time: 30 IO/s: 106829 MB/s: 417

root@megamini:/home/rz # nvmecontrol perftest -n 32 -o read -s 131072 -t
30 nvme0ns1
Threads: 32 Size: 131072 READ Time: 30 IO/s: 13783 MB/s: 1722

root@megamini:/home/rz # nvmecontrol perftest -n 64 -o read -s 4096 -t
30 nvme0ns1
Threads: 64 Size: 4096 READ Time: 30 IO/s: 106826 MB/s: 417

root@megamini:/home/rz # nvmecontrol perftest -n 64 -o read -s 131072 -t
30 nvme0ns1
Threads: 64 Size: 131072 READ Time: 30 IO/s: 12628 MB/s: 1578

Still, good old "dd" extracts more throughput from poor device:

root@megamini:/home/rz # dd if=/dev/nvd0 of=/dev/null bs=819200
iflag=direct status=progress
^C179769344000 bytes (180 GB, 167 GiB) transferred 70.002s, 2568 MB/s
222633+0 records in
222633+0 records out
182380953600 bytes transferred in 70.986171 secs (2569246265 bytes/sec)

iostat during dd running shows:

root@megamini:~ # iostat -w 1 -d nda0
nda0 
KB/t tps MB/s 
398 70 27.37 
400 6665 2604 
400 6681 2610 
400 6681 2610 
400 6680 2609 
400 6678 2609 
400 6651 2598 
400 6666 2604 
400 6672 2606 

From above, it seems dd splits reads into 400K blocks, so let's try
"nvmecontrol perftest" again with 512K block:

root@megamini:/home/rz # nvmecontrol perftest -n 4 -o read -s 524288 -t
30 nvme0ns1
Threads: 4 Size: 524288 READ Time: 30 IO/s: 5059 MB/s: 2529

root@megamini:/home/rz # nvmecontrol perftest -n 16 -o read -s 524288 -t
30 nvme0ns1
Threads: 16 Size: 524288 READ Time: 30 IO/s: 5752 MB/s: 2876

root@megamini:/home/rz # nvmecontrol perftest -n 32 -o read -s 524288 -t
30 nvme0ns1
Threads: 32 Size: 524288 READ Time: 30 IO/s: 6016 MB/s: 3008

root@megamini:/home/rz # nvmecontrol perftest -n 64 -o read -s 524288 -t
30 nvme0ns1
Threads: 64 Size: 524288 READ Time: 30 IO/s: 6002 MB/s: 3001

Now I see it's getting closer to 300km/h Autobahn, and it's totally
possible to extract good performance from this cheap device. ;-)

By the way, on my laptop I have SAMSUNG MZALQ256HBJD-00BL2, that is also
connected at 8GT/s x4, but the throughput is three times as low. This
laptop runs FreeBSD 13.5-p13. Looks like the nvme driver really got some
performance gain over the time. ;-) Thank you for your work!

---
Regards,
Ruslan