Is it me or is FreeBSD slower on Xen than Linux?

[Roger Pau Monné]

On Tue, Aug 16, 2016 at 11:29:43AM +0200, rainer at ultra-secure.de wrote:
> Am 2016-08-16 10:54, schrieb Roger Pau Monné:
> > On Mon, Aug 15, 2016 at 05:54:52PM +0200, rainer at ultra-secure.de wrote:
> > > Hi,
> > > 
> > > I've got a problem.
> > > 
> > 
> > Hello,
> > 
> > > 
> > > For a customer, I run a VM in Xen that should perform a certain task
> > > in PHP
> > > (written using the ZendFrameWork).
> > > 
> > > That task takes about 18-20 seconds on FreeBSD 10.3 amd64, MariaDB
> > > 5.5.0,
> > > php 5.5.37 in a VM that has 8 vCPUs and 16GB of memory
> > > The "reference" server that the customer uses is somewhere else and
> > > manages
> > > to perform the same task in 3s.
> > > 
> > > I've tried this with FreeBSD 10.3, PHP7.0 and MariaDB 10.1 and it
> > > takes
> > > about 9s.
> > 
> > In the sentence above, are you running it in a Xen VM or on bare metal?
> > 
> 
> This is both Xen.
> I think the customer is also running it on some sort of virtualization.

Hm, so a given workload on Xen takes ~9s, and it also takes ~9s when run on 
bare metal FreeBSD, is that right?

> 
> > > I've tried it on physical hardware with 10.3, PHP5.5, MariaDB 5.5
> > > and it
> > > also takes about 9s (that machine hosts a load of other sites but
> > > has lot of
> > > cores and memory available).
> > > 
> > > 
> > > Then, I've installed an Ubuntu 14 VM in XenServer. It comes with
> > > PHP5.5 and
> > > MariaDB 5.5 by default. It's VM with 2vCPUs and 8GB RAM.
> > > 
> > > There, the script take about 9s, too (just as if it was running on
> > > physical
> > > FreeBSD).
> > 
> > I'm not sure I understood your problem right, is it that FreeBSD on Xen
> > always takes 18-20s to perform a task while on bare metal it only takes
> > ~9s?
> 
> 
> Well, that in itself is not the problem.
> The problem is that Linux on Xen is as fast as my bare metal (incidentally,
> both the physical Xen host (Dom0) and the physical server I ran the script
> for comparison are the same hardware).

But from your description above I take that you get the same performance 
when running FreeBSD on Xen or when running FreeBSD on bare metal, in which 
case I'm not sure if this problem is related to Xen at all.

> 
> > If that's the case, I would recommend that you first try to disable PV
> > disks
> > and nics, by adding the following to your /boot/loader.conf:
> > 
> > hw.xen.disable_pv_disks=1
> > hw.xen.disable_pv_nics=1
> 
> OK, can I still boot the VM with this or will NICs and disks show up as
> different devices then?

NIC will show up as "re", disks as "ada" (which is what you already have).

> 
> > If that still yelds the same performance (or worse), then you could
> > still
> > try to disable all Xen code, by removing:
> > 
> > options 	XENHVM
> > device		xenpci
> > 
> > From you kernel config and recompiling the kernel.
> 
> I'm using stock FreeBSD 10.3.
> I was under the assumption that this is the "optimal" configuration.

It should be, I'm just trying to figure out if there's something there 
that's hampering performance. But please read above because it's still not 
clear to me that this is related to Xen.

Roger.