NVMe performance 4x slower than expected

[Tobias Oberstein]

>> I was advised (off list) to run tests against a pure ramdisk.
>>
>> Here are results from a single socket E3:
>>
>> https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_ramdisk.md#xeon-e3-machine
>>
>> and here are results for the 48 core box
>>
>> https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_ramdisk.md#48-core-big-machine
>>
>> Performance with this box is 1/10 on this test compared to single socket E3!
>>
>> Something is severely wrong. It seems, there might be multiple issues
>> (not only NVMe). And this is after already running with 3 patches to
>> make it even boot.
> The speed of dd already looks wrong.

Yes.

>
> Check the CPU frequency settings in BIOS, and check what sysctl dev.cpu
> reports.  Ensure that cpufreq.ko is loaded from loader.conf.

It's loaded now, and CPU clock is at maximum: dev.cpu.0.freq: 3000

Unfortuanetly, performance (randomio/fio) did not change ..