SSDs peformance on head/freebsd-10 stable using FIO

[Kashyap Desai]

Hi Motin,



I am trying to collect IOPs and throughput using FIO on FreeBSD-10-stable
as below post mentioned that CAM can reach upto 1,000,000 IOPS using
Fine-Grained CAM locking.



http://www.freebsd.org/news/status/report-2013-07-2013-09.html#GEOM-Direct-Dispatch-and-Fine-Grained-CAM-Locking



I am using below FIO parameter.



[global]

ioengine=posixaio

buffered=0

rw=randread

bs=4K

iodepth=32

numjobs=2

direct=1

runtime=60s

thread

group_reporting=1

[job1]

filename=/dev/da0

[job2]

filename=/dev/da1

[job3]

filename=/dev/da2

[job4]

filename=/dev/da3

[job4]

filename=/dev/da4

..



I have 8 SSDs in my setup and all 8 SSDs are behind LSI’s 12Gp/s MegaRaid
Controller as JBOD. I also found FIO can be used in Async mode after
loading “aio” kernel module.

Using single SSD, I am able to see  110K-130K IOPs.  This IOPs counts are
matching with what I see on Linux machine.



Now, I am not able to scale IOPs on my machine after 200K.  I see CPU is
almost occupied and no idle time after IOPs reach to 200K.





If you have any pointers to try with,  I can do some experiment on my setup.





Thanks, Kashyap