Attempting to Get Xen FreeBSD Dom0 working
Miguel Clara
miguelmclara at gmail.com
Mon Dec 15 01:26:39 UTC 2014
On Mon, Dec 15, 2014 at 1:24 AM, Miguel Clara <miguelmclara at gmail.com>
wrote:
>
>
> On Mon, Dec 15, 2014 at 12:30 AM, Miguel Clara <miguelmclara at gmail.com>
> wrote:
>>
>>
>> On Sun, Dec 14, 2014 at 11:34 PM, David P. Discher <dpd at dpdtech.com>
>> wrote:
>>>
>>> First make sure you CPU supports both EPT and has a IOMMU.
>>>
>>
>> Yes it does.
>>
>> I should note theat I've already tested with Xen in this laptop using
>> Debian.
>>
>>
>>>
>>> It sounds like you have Xen to use the serial console.
>>>
>>> To /etc/ttys, you need to add:
>>>
>>> xc0 "/usr/libexec/getty Pc" vt100 on secure
>>>
>>> Also, make sure all the ttyu[0-4] are off. Recent versions of FreeBSD
>>> maybe putting onifconsole … and I can't remember all the combinations I
>>> used, but I had a lot of issues with Xen and FreeBSD fighting for the
>>> serial ports.
>>>
>>
>> Those this make sense:
>> [miguelc at hpbsd]~% cat /etc/ttys|eg 'ttyuv|xc0'
>> xc0 "/usr/libexec/getty Pc" vt100 on secure
>> [miguelc at hpbsd]~% cat /etc/ttys | eg 'ttyu|xc0'
>> ttyu0 "/usr/libexec/getty 3wire" vt100 onifconsole secure
>> ttyu1 "/usr/libexec/getty std.9600" dialup off secure
>> ttyu2 "/usr/libexec/getty std.9600" dialup off secure
>> ttyu3 "/usr/libexec/getty std.9600" dialup off secure
>> xc0 "/usr/libexec/getty Pc" vt100 on secure
>>
>>
>> (I've just added xc0 following the guide, not sure if ttyu0 onifconsole
>> is correct or should just be off)
>>
>>>
>>> Also, make sure /boot/loader.conf is configured for console=“vidconsole"
>>> on the freebsd side, and that the xen_cmdline should have “console=vga” and
>>> not =com1. It’s not explicit, but the Xen kernel creates the “console”.
>>> The xc0 device (and Xen itself) seems to take care of piping the Dom0
>>> console over to whatever Xen is using as console.
>>>
>>
>> Oh wait... com1 and vga ... I copy pasted from the guide and left it to
>> "com1".. this must be the issue, let me retry and I'll post some feedback.
>>
>>>
>>>
>>> Other tips, if you are running ZFS, you will probably need to add
>>> "vm.max_wired=-1” to /etc/sysctl.conf (I’m actually not sure this is valid,
>>> but if you don’t you’ll run out of wired memory and all the “xl” commands
>>> will fail. Or limit the size of ARC. )
>>>
>>
>> My arc_max is limitted to 2G atm but in any case I've set the
>> vm.max_wired to -1 and as you say we will see.
>>
>>
>>>
>>> You might need to turn off MSI interrupts on AHCI, "hint.ahci.0.msi=0”
>>> in /boot/loader.conf. However, try both ways (default I believe is =1).
>>> I’m running Intel ICH10 and have to disable MSI. Roger has ICH8 and
>>> doesn’t seem to have the issue.
>>>
>>> Do I have a way to see what ICH is it in freebsd... (I can ofc lookit up
>> online) its a I7 so I think its ICH10, but not sure.
>>
>>
>>> The latest issue is very poor network performance (with Intel 82574L)
>>> from the DomU (guests) over the bridge and to the network. However, Guests
>>> -> Dom0 seem ok.
>>>
>>>
>>> Console issue solved, however I'm getting a panic message with
>
> Presently, iommu must be enabled for pvh
>
> Looking the model up online I see:
> Intel® Virtualization Technology (VT-x) ‡ Yes
> Intel® Virtualization Technology for Directed I/O (VT-d) ‡ No
> Intel® VT-x with Extended Page Tables (EPT) ‡ Yes
>
> So I guess it does have EPT and vtx but not vtd, so I don't have ioemu :(
>
*IOMMU
>
>
> -
>>> David P. Discher
>>> http://davidpdischer.com/
>>> AIM: DavidDPD | Y!M: daviddpdz
>>>
>>>
>>>
>>> On Dec 13, 2014, at 10:51 PM, Miguel Clara <miguelmclara at gmail.com>
>>> wrote:
>>>
>>> > I was just trying too boot Freebsd Xen dom0 on a laptop but I just
>>> get a black screen after the boot process....
>>> >
>>> > any idea what it could be?
>>> >
>>> > the system boots fine wihtout loading "boot/xen", I'm not sure how to
>>> get more info with the black scren!
>>> >
>>> > thanks
>>> >
>>> >
>>> > Melhores Cumprimentos // Best Regards
>>> > -----------------------------------------------
>>> > Miguel Clara
>>> > IT - Sys Admin & Developer
>>> > E-mail: miguelmclara at gmail.com
>>> > <linkedin.png> www.linkedin.com/in/miguelmclara/
>>> >
>>> > On Tue, Dec 9, 2014 at 7:17 PM, David P. Discher <dpd at dpdtech.com>
>>> wrote:
>>> > ah, sorry missed that. Will try that today.
>>> >
>>> > AHCI lasted over night with MSI off. Something I noticed, is that
>>> when the AHCI bus was timing out, it looked like the Xen Kernel was
>>> re-scanning the PCI bus. (Sorry, didn’t save these logs). I’ve love to dig
>>> into this further.
>>> >
>>> > Please let me know what/where to add some debugging code.
>>> >
>>> > -
>>> > David P. Discher
>>> > http://davidpdischer.com/
>>> > AIM: DavidDPD | Y!M: daviddpdz
>>> > Mobile: 408.368.3725
>>> >
>>> >
>>> >
>>> > On Dec 9, 2014, at 12:27 AM, Roger Pau Monné <roger.pau at citrix.com>
>>> wrote:
>>> >
>>> > > Hello,
>>> > >
>>> > > El 08/12/14 a les 23.45, David P. Discher ha escrit:
>>> > > <SNIP>
>>> > >>
>>> > >> Sent SIGTERM to all processes
>>> │
>>> > >> Sent SIGKILL to all
>>> processes───────────────────────────────────────────────┘
>>> > >> Requesting system reboot
>>> > >> [ 1157.299205] Restarting system.
>>> > >> root at borg:/zdata/debian #
>>> > >> root at borg:/zdata/debian #
>>> > >> root at borg:/zdata/debian # xl create -c debian.cfg
>>> > >> root at borg:/zdata/debian # xl destroy debian
>>> > >> xc: error: Could not bounce buffer for version hypercall (35 =
>>> Resource temporarily unavailabl): Internal error
>>> > >> xc: error: Could not bounce buffer for version hypercall (35 =
>>> Resource temporarily unavailabl): Internal error
>>> > >> xc: error: Could not bounce buffer for version hypercall (35 =
>>> Resource temporarily unavailabl): Internal error
>>> > >> xc: error: Could not bounce buffer for version hypercall (35 =
>>> Resource temporarily unavailabl): Internal error
>>> > >> xc: error: Could not bounce buffer for version hypercall (35 =
>>> Resource temporarily unavailabl): Internal error
>>> > >> xc: error: Could not bounce buffer for version hypercall (35 =
>>> Resource temporarily unavailabl): Internal error
>>> > >> libxl: error: libxl.c:658:libxl_list_domain: getting domain
>>> info list: Resource temporarily unavailable
>>> > >> debian is an invalid domain identifier (rc=-5)
>>> > >> root at borg:/zdata/debian #
>>> > >>
>>> > >> I’m running AHCI with MSI off in the FreeBSD kernel, and so far, so
>>> good on that front. The great thing is now I got the Xen console working,
>>> so can get the debug output. However the bounce buffer/hypercall issue i
>>> would think is far more important than MSI interrupts at the monument.
>>> > >
>>> > > Glad to know you got it working at the end! I've already pointed this
>>> > > out in my last email, but did you try to increase vm.max_wired even
>>> further?
>>> > >
>>> > > This usually happens when mlock in
>>> > > freebsd_privcmd_alloc_hypercall_buffer (xc_freebsd_osdep.c) fails to
>>> > > wire down the memory that would be used by the hypercalls.
>>> > >
>>> > > Roger.
>>> > >
>>> >
>>>
>>>
More information about the freebsd-xen
mailing list