Re: Call for testers with NVMe
- In reply to: Warner Losh : "Re: Call for testers with NVMe"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Tue, 08 Sep 2026 13:14:08 UTC
Seen no difference. I used two Gen4 drives, Samsung 990 PRO and Crucial T500: old = D59151-52, new = +D59153-54, arb = +D59155 990 PRO (RAB 2) old new arb perftest 4k x4 IO/s 694k 688k 694k perftest 128k x1 MB/s 5547 5557 5563 fio 4k QD32 IOPS 651k 626k 625k fio 4k QD1 psync IOPS 29.5k 29.1k 29.2k fio seq 1m QD8 MiB/s 4960 4954 5020 T500 (RAB 6) old new arb perftest 4k x4 IO/s 468k 472k 474k perftest 128k x1 MB/s 4861 4873 4876 fio 4k QD32 IOPS 339k 340k 339k fio 4k QD1 psync IOPS 14.9k 14.9k 14.9k fio seq 512k QD8 MiB/s 2661 2665 2666 Raw logs available on request. On Tue, Sep 08, 2026 07:03 AM, Warner Losh <imp@bsdimp.com> wrote: > So we have a variety of NVMe drives with non-zero ARB values, but we're > still seeing line rate or datasheet maximums on our streaming workload... > > What workload would show an improvement? > > I'd really prefer to get a chance to test these on Netflix workloads > before you land it (if it's not too late). > > Warner > > On Thu, Sep 3, 2026 at 7:07 AM Abdelkader Boudih <seuros@freebsd.org> > wrote: > >> Thanks for testing. >> >> This shows a 47% improvement , 780 MB/s to 1150 MB/s in sequencial output >> and slight increase in the random seek. >> >> Linux numbers are useful but that was tested against another filesystem. >> >> Also, your 2 kernels are from July 12 and Sep 3 , So that proof that >> when people will upgrade from 15.x , they will get an increase that was not >> only tested in 1 tester. >> >> I will wait for Olivier's response before we land these. (I already have >> more changes that i'm trying to benchmark here). >> >> Best, >> Abdelkader >> >> On Thu, 3 Sep 2026, at 12:12, Martin Cracauer wrote: >> >> Abdelkader Boudih wrote on Wed, Sep 02, 2026 at 10:41:25AM +0100: >> > Hi all, >> > >> > I am about to land the D59156 stack in about 24 hours: >> https://reviews.freebsd.org/D59156 >> > >> > It is a stack of 6 commits, D59151 --> D59156. >> > >> > This stack allows NVMe drives to run at full speed and use all >> available lanes, instead of being quirked to use only one. >> >> Here are my benchmarks. Machines are Thinkpad T14 AMD gen 1 with 48 >> GB RAM and 1 TB NVMe WD black sn850x. >> >> sn850x 1TB on -current ufs: main-n287387-7aa34e12c9e1 Jul 12, 2026` >> ---Sequential Output----- ---Input---- ------Random-- >> -----Random---- >> ---Block---- ---Rewrite-- ---Block---- ---ro Seeks--- -Seeks >> rewrite- >> MB M/sec %CPU M/sec %CPU M/sec %CPU /sec %CPU /sec >> %CPU >> 65536 780.7 33.8 779.4 38.2 2301.4 53.6 73650.7 32.1 30248.5 >> 25.3 >> cseekbench: 126.585791 usec/access 0.030 GB/sec >> main-n288812-799061fc769c Sep 3: >> MB M/sec %CPU M/sec %CPU M/sec %CPU /sec %CPU /sec >> %CPU >> 65536 1150.6 49.2 776.9 40.5 2229.2 60.5 86489.4 37.8 33537.9 >> 28.0 >> cseekbench: 127.185498 usec/access 0.030 GB/sec >> >> Linux 6.17.0-rc2 ext4fs: >> ---Sequential Output----- ---Input---- ------Random-- >> -----Random---- >> ---Block---- ---Rewrite-- ---Block---- ---ro Seeks--- -Seeks >> rewrite- >> MB M/sec %CPU M/sec %CPU M/sec %CPU /sec %CPU /sec >> %CPU >> 65536 2693.9 95.4 714.3 37.7 2786.7 63.0 44818.0 30.1 26286.2 >> 23.5 >> MB M/sec %CPU M/sec %CPU M/sec %CPU /sec %CPU /sec >> %CPU >> 65536 1630.2 96.7 628.4 56.6 2520.4 71.7 41362.8 49.8 24476.6 >> 36.8 >> cseekbench: 87.998063 usec/access 0.043 GB/sec >> Linux grisu 6.8.0-137-generic: >> MB M/sec %CPU M/sec %CPU M/sec %CPU /sec %CPU /sec >> %CPU >> 65536 1133.0 100.4 742.4 80.7 1831.7 99.0 39877.3 46.4 24268.6 >> 40.4 >> cseekbench: 118.671478 usec/access 0.032 GB/sec >> Linux grisu 7.2.3: >> MB M/sec %CPU M/sec %CPU M/sec %CPU /sec %CPU /sec >> %CPU >> 65536 1562.6 97.5 617.6 57.9 2466.8 74.2 41675.5 50.3 23832.1 >> 39.2 >> cseekbench: 87.090687 usec/access 0.044 GB/sec >> >> >> Benchmarks are here: >> https://github.com/cracauer/bench-disk-bonnie (pre-GPL fork of bonnie) >> https://github.com/cracauer/cseekbench >> >> Martin >> -- >> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% >> Martin Cracauer <cracauer@cons.org> http://www.cons.org/cracauer/ >> >> >>