Any ideal way to run FIO benchmarking for NVMEe devices in FreeBSD

[Enji Cooper]

> On Feb 22, 2019, at 01:29, Rajesh Kumar <rajfbsd at gmail.com> wrote:
> 
> Hi Enji Cooper,
> 
> I am using Samsung 960 PRO
> 
> https://www.samsung.com/semiconductor/minisite/ssd/product/consumer/960pro/

Hi Rajesh,
    I asked about the datasheet, because there might be some hints in terms of the number of parallel jobs you might want to apply as well as the I/O queue depth, which will affect the performance of the drive. Otherwise you’ll be throwing values against a wall, seeing what will stick, which is sort of ok if you bound your testing and adjust based on performance, but if your initial hunch is off, it can mislead you.
    Similarly, check to see if there are any tunables or sysctls that will bound the device in terms of the queue depth and I/O requests.
    As others have noted, test the device directly if you want to know its raw performance. Only test with a filesystem if your intent is to see how the device will perform with a filesystem on it (and disable filesystem sync if you want to test max throughput with the overhead of a filesystem). Testing with a filesystem can reveal some potentially interesting characteristics, in terms of limitations in VFS / the filesystem implementation, which might be helpful if you’re trying to determine why there’s a difference between raw device speed and the speed with a filesystem on it. Testing with the same file in different directories is ok, as long as you blow the drive cache—which will have a noticeable performance impact on conventional (PMR, SMR, etc) drives, as you want to test the worst case speed of the device, instead of the cache. It should matter a bit less with faster devices like SSDs/NVMe drives. Testing with files in the same lateral filesystem hierarchy (same directory), might reveal issues with filesystem locking (directory inode performance), but that shouldn’t be the primary goal of your testing. It’s just something to keep in mind.
    Happy testing!
HTH,
-Enji