Disk performance on ESXi with FreeBSD 7.1

[Scott Long]

Sebastiaan van Erk wrote:
> Hi,
> 
> I'm running FreeBSD on ESXi but I'm having serious issues with disk 
> performance, and I'm wondering if it might have something to do with the 
> scsi driver (or the virtual hardware not returning proper values for its 
> capabilities or something)..
> 
> I have both a FreeBSD-amd64 and Linux Ubuntu 8.10-amd64 virtual machine 
> (8GB disk, 512MB RAM, 2-CPU) and run dbench on both of them. The linux 
> machine is out of the box, not optimized for vmware, 
> VMI/paravirtualization is off, as is VMotion. The results for dbench <n> 
> are as follows:
> 
> <n>             1               2               4
> freebsd         12.0009         13.6348         12.9402         (MB/s)
> linux           376.145         651.314         634.649         (MB/s)
> 
> Thus there is approx a factor 30 difference for dbench 1, and I cannot 
> imagine linux being that much faster just due to some performance tuning 
> kernel parameters.
> 
> I tried both the VMware LSI Logic controller and the BusLogic 
> controller. Here is the relevant dmesg output of both:
> 
> LSI:
> mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1080-0x10ff mem 
> 0xf4810000-0xf4810fff irq 17 at device 16.0 on pci0
> mpt0: [ITHREAD]
> mpt0: MPI Version=1.2.0.0
> da0 at mpt0 bus 0 target 0 lun 0
> da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device
> da0: 3.300MB/s transfers
> da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C)
> 
> BusLogic:
> bt0: <Buslogic Multi-Master SCSI Host Adapter> port 0x1060-0x107f mem 
> 0xf4810000-0xf481001f irq 17 at device 16.0 on pci0
> bt0: BT-958 FW Rev. 5.07B Ultra Wide SCSI Host Adapter, SCSI ID 7, 192 CCBs
> bt0: [GIANT-LOCKED]
> bt0: [ITHREAD]
> da0 at bt0 bus 0 target 0 lun 0
> da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device
> da0: 40.000MB/s transfers (20.000MHz DT, offset 15, 16bit)
> da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C)
> 
> Something that I noticed was the extremely slow transfer rates mentioned 
> with the da0 device.
> 
> When I'm running dbench the server is not very busy:
> 
> CPU:  0.2% user,  0.0% nice,  6.4% system,  0.7% interrupt, 92.7% idle
>  1172 root        1  -8    0  4604K  1228K biowr  1   0:41  4.98% dbench
> 
> I really want to get this working because I want to run a big production 
> site on FreeBSD. But currently the disk speed is just unworkable.
> 
> I was wondering if anybody had any ideas about how to get proper disk 
> speeds on FreeBSD, making it a viable guest operating system.
> 
> If any other info is needed, I'm willing to invest quite some time to 
> provide it!
> 
> Regards,
> Sebastiaan

Run the following command:

sudo camcontrol tags da0

If it returns something like this:

(pass0:mpt0:0:0:0): device openings: 1

then run the following command:

sudo camcontrol tags da0 -N 64

If this works in improving performance, it can be put into a startup
script.  I have no idea why the controller is misbehaving with this yet,
but I'm working on it.

Scott