Network problems while running VirtualBox

[Peter Ross]

Quoting "Gustau Pérez" <gperez at entel.upc.edu>:

> Al 19/07/2011 08:40, En/na Peter Ross ha escrit:
>> Quoting "Gustau Pérez" <gperez at entel.upc.edu>:
>>
>>> Al 13/07/2011 23:57, En/na Peter Ross ha escrit:
>>>> I have a problem with the network while running VirtualBox.
>>>>
>>>> As soon as I _run_ a VirtualBox I am not able to copy large files  
>>>> (e.g. virtual disks or ZFS snapshots) using ssh/scp to another  
>>>> machine.
>>>>
>>>> The ssh crashes with "Write failed: Cannot allocate memory"
>>>>
>>>> thrown by a write(2) in /usr/src/crypto/openssh/roaming_common.c  
>>>> (in function roaming_write). It returns the ENOMEM (an error it  
>>>> should never return, according to the mainpage;-)
>>>>
>>>> It is immediately working when I stop the VirtualBox, even if the  
>>>> VirtualBox kernel modules are still loaded.
>>>> ..
>>>> I experienced the problem with VirtualBox 3.2 first but the  
>>>> upgrade to VirtualBox 4.0.8 and the base system recently did not  
>>>> help.
>>>
>>>   I have AMD64/STABLE+virtualbox[4.0.10|4.1Beta] on a Dell R710  
>>> with two scenarios:
>>>
>>>    1.- Sending large files from the host to the guest with scp.
>>>    1,.  rsync+ssh in the host machine (with virtualbox) sending  
>>> large files to a remote machine.
>>>
>>>   in fact both scenarios are the same, the host machine sending  
>>> large chunks of data. Both fail as it fails to everyone in the list.
>>>
>>>   Ok. So far so good. To track down the problem I tested may  
>>> changes of configuration with STABLE (if you want the details  
>>> please let me know and I'll send them to the list). None did the  
>>> trick. I even tried communications between the host and the guest  
>>> with vboxnet. scp failed with the "no memory allocation" problem.
>>>
>>>   Now I tried AMD64/CURRENT+virtualbox 4.0.10 on a laptop. I  
>>> tested an scenario like the one described in number 1. It worked  
>>> just fine. In fact I tried something like this in the host machine  
>>> just to be sure:
>>>
>>>      # for i in {1..10}
>>>               do
>>>                     cat "large_file.data" | ssh -l root  
>>> 192.168.56.101 "cat -> /dev/null"
>>>               done
>>>
>>>    Where 192.168.56.101 is the guest machine (I'm using vboxnet).  
>>> This large file is 8 Gb file. So it gave me an 80Gb transfer. It  
>>> worked fine. This scenario would have failed with STABLE.
>>>
>>>   So I guess it has something to do with the combo  
>>> STABLE+Virtualbox. There must a change in CURRENT that doesn't  
>>> trigger the problem. I think it would be appropriate if anyone  
>>> else could also try with CURRENT.
>>>
>>>   As an addition I remember all of this worked with 8.1 and an  
>>> early version of virtualbox 3.2 series . But I can't say which  
>>> revision of 8.1 I had when I was using that kind of scenarios.
>>
>> I experienced the problem first on VirtualBox 3.2 and  
>> FreeBSD-8.2-PRERELEASE. Marlon already in September 2010.
>>
>> The laptop isn't the ultimate test as long as you haven't the same  
>> data going through the netgraph subsystem.
>>
>> Can you try to set net.graph.maxdata as well (see my other e-mail).  
>> Does it solve your problem too?
>>
>> Thanks for your help
>> Peter
>>
>>
>
>    Hi,
>
>    My test system is a FreeBSD8.2/AMD64 Stable r222508 in a DELL  
> R710 with 24GB of RAM with a dual Intel PRO/1000 and 4 Broadcom  
> Extreme II BCM5709. The broadcoms are lagged together giving bridged  
> connectivity to the virtual machines to the real world. I'm also  
> using a vboxnet network to bring a dedicated network between the  
> virtual machines and the host system.
>
>   I've been testing with net.graph.maxdata="65536". I've been able  
> to do large transfers (like 25GB with rsync and scp) from the host  
> to a remote system. Also I've been able to do transfers of about  
> 10GB from the host system to one guest system and they went well.  
> Without setting net.graph.maxdata I usually triggered the problem  
> with transfers of about 1 or 2GB. As you can see, I saw no problems  
> with the real network nor with the virtual network.
>
>   I did not report earlier because I wanted to be sure enough. Now I  
> would say that maxdata removed the problem for me.
>
>   What I do not get is why it works with CURRENT having  
> net.graph.maxdata="256".

This is tested on the laptop? Are you able to upgrade the Dell to  
-current to confirm? (Or did you do it already?)

> Anyway I think it would be nice to know why setting the maxdata  
> solves the problem because it would allow  the virtualbox team to  
> add a note in pkg-message with the appropriate explanation or even  
> proposing a PR to STABLE to fix the issue.

Of course 65536 is a very high value. I just used it as a starting  
point to test but I planning to decrease it gradually next week.

Are your VirtualBoxes busy "network-wise"?

My one is. It is a Zimbra mail server running on Red Hat Enterprise  
Linux (the only inhouse Linux server I could not convert to native  
FreeBSD but I did not want it to give an extra server on its own..)

It serves 50 accounts, and gets SMTP traffic (spam) arriving, Outlook  
connections using HTTP, a web interface, internal redirections to  
spamassassin and amavis etc.

The buffers are limited by default to avoid general kernel memory  
shortage I guess. There is some advice related to network tuning (e.g.  
http://www.freebsdonline.com/content/view/49/63/ )

But I (and others, as it looks like) did not see the reason why it  
failed. You could call it a pilot error if I did not consider the  
network load as an issue. In the beginning I saw the scp as an  
isolated process and wondered why it failed (the host system isn't  
really busy).

It is probably not really a VirtualBox issue at all. It is just  
VirtualBox is "the box" that hides all that's going on, it is always  
good to remember that the load is still there.

I had issues in the past with VMWare as well. It was difficult for  
people to understand "why it's so slow" when grunty boxes were  
"subdivided" by developers in many many virtual servers. The combined  
load was very hard to grasp, it is difficult to have all relevant data  
available when an intermittent problem occurs. We had VMWare  
consultants called in by the management who walked away after a day  
with "the problem does not exist"...

Regards
Peter