Any ideal way to run FIO benchmarking for NVMEe devices in FreeBSD

[Rajesh Kumar]

Thanks Enji Cooper, Warner Losh, Alan Somers and Rebecca Cran for your
Inputs.

I did some test and here's my observation.
1) iogine "posixaio" (or) "sync" - no much difference. Is this expected?
2) As said, raw device gives better numbers compared to filesystem.
3) With "thread" option, I see the numbers going considerably down. Is this
expected?
4) I see good numbers for IOPS, but the MB/s is less comparatively. Any
reasons?

Enji Cooper, the mentioned link has all the documents (data sheet, user
guide etc.,). Not sure if you have seen them. Anyway, I will consider about
the points you have mentioned in my testing and see how they impact the
performance. Thanks.

Thanks,
Rajesh.

On Fri, Feb 22, 2019 at 9:24 PM Enji Cooper <yaneurabeya at gmail.com> wrote:

>
> On Feb 22, 2019, at 01:29, Rajesh Kumar <rajfbsd at gmail.com> wrote:
>
> Hi Enji Cooper,
>
> I am using Samsung 960 PRO
>
> https://www.samsung.com/semiconductor/minisite/ssd/product/consumer/960pro/
>
>
> Hi Rajesh,
>     I asked about the datasheet, because there might be some hints in
> terms of the number of parallel jobs you might want to apply as well as the
> I/O queue depth, which will affect the performance of the drive. Otherwise
> you’ll be throwing values against a wall, seeing what will stick, which is
> sort of ok if you bound your testing and adjust based on performance, but
> if your initial hunch is off, it can mislead you.
>     Similarly, check to see if there are any tunables or sysctls that will
> bound the device in terms of the queue depth and I/O requests.
>     As others have noted, test the device directly if you want to know its
> raw performance. Only test with a filesystem if your intent is to see how
> the device will perform with a filesystem on it (and disable filesystem
> sync if you want to test max throughput with the overhead of a filesystem).
> Testing with a filesystem can reveal some potentially interesting
> characteristics, in terms of limitations in VFS / the filesystem
> implementation, which might be helpful if you’re trying to determine why
> there’s a difference between raw device speed and the speed with a
> filesystem on it. Testing with the same file in different directories is
> ok, as long as you blow the drive cache—which will have a noticeable
> performance impact on conventional (PMR, SMR, etc) drives, as you want to
> test the worst case speed of the device, instead of the cache. It should
> matter a bit less with faster devices like SSDs/NVMe drives. Testing with
> files in the same lateral filesystem hierarchy (same directory), might
> reveal issues with filesystem locking (directory inode performance), but
> that shouldn’t be the primary goal of your testing. It’s just something to
> keep in mind.
>     Happy testing!
> HTH,
> -Enji
>