Re: ZFS + FreeBSD XEN dom0 panic

On 2022.03.09. 10:42, Roger Pau Monné wrote:
> On Sun, Mar 06, 2022 at 02:41:17PM +0200, Ze Dupsys wrote:
>> Then i caught another sysctl variable that is growing due to XEN,
>> "kern.msgbuf: Contents of kernel message buffer". I do not know how this
>> variable grows or by which component it is managed, but in VM start/stop
>> case it grows and contains lines with pattern like so:
>> ..
>> xnb(xnb_rxpkt2rsp:2059): Got error -1 for hypervisor gnttab_copy status
>> xnb(xnb_ring2pkt:1526): Unknown extra info type 255.  Discarding packet
>> xnb(xnb_dump_txreq:299): netif_tx_request index =0
>> xnb(xnb_dump_txreq:300): netif_tx_request.gref  =0
>> xnb(xnb_dump_txreq:301): netif_tx_request.offset=0
>> xnb(xnb_dump_txreq:302): netif_tx_request.flags =8
>> xnb(xnb_dump_txreq:303): netif_tx_request.id    =69
>> xnb(xnb_dump_txreq:304): netif_tx_request.size  =1000
>> xnb(xnb_dump_txreq:299): netif_tx_request index =1
>> xnb(xnb_dump_txreq:300): netif_tx_request.gref  =255
>> xnb(xnb_dump_txreq:301): netif_tx_request.offset=0
>> xnb(xnb_dump_txreq:302): netif_tx_request.flags =0
>> xnb(xnb_dump_txreq:303): netif_tx_request.id    =0
>> xnb(xnb_dump_txreq:304): netif_tx_request.size  =0
>> ..
>>
>> Those lines in that variable just keep growing and growing, it is not that
>> they are flushed, trimmed or anything. Each time i get the same message on
>> serial output, it has one more section of error appended to "same-previous"
>> serial output message and sysctl variable as well. Thus at some point serial
>> output and sysctl contains a large block of those errors while VM is
>> starting. So at some point the value of this sysctl could be reaching max
>> allowed/available and this makes the system panic.  I do not know the reason
>> for those errors, but actually if there was a patch to suppress them, this
>> could be "solved". Another diff chunk might be related to this:
>> +dev.xnb.1.xenstore_peer_path: /local/domain/7/device/vif/0
>> +dev.xnb.1.xenbus_peer_domid: 7
>> +dev.xnb.1.xenbus_connection_state: InitWait
>> +dev.xnb.1.xenbus_dev_type: vif
>> +dev.xnb.1.xenstore_path: backend/vif/7/0
>> +dev.xnb.1.dump_rings:
>> +dev.xnb.1.unit_test_results: xnb_rxpkt2rsp_empty:1765 Assertion Error:
>> nr_reqs == 0
>> +xnb_rxpkt2rsp_empty:1767 Assertion Error: memcmp(&rxb_backup,
>> &xnb_unit_pvt.rxb, sizeof(rxb_backup)) == 0
>> +xnb_rxpkt2rsp_empty:1769 Assertion Error: memcmp(&rxs_backup,
>> xnb_unit_pvt.rxs, sizeof(rxs_backup)) == 0
>> +52 Tests Passed
>> +1 Tests FAILED
> So you have failed tests for netback. Maybe the issue is with
> netback rather than blkback.

Just found that there are no errors with network, most probably. Based 
on https://reviews.freebsd.org/D9234?id=24172#201024, since i was 
monitoring sysctl variables with 'sysctl -a', most probably that did 
"sysctl dev.xnb.0.unit_test_results" implicitly for each xnb interface, 
thus the extra output, since reading value actually runs tests. So now i 
upgraded my monitoring command to 'sysctl -a -N'. Never did i expect 
that sysctl variable reading could trigger test suite execution.

I did not understand from comments though - how do you instantiate xnb 
interface?

"ifconfig xnb create" returned:
ifconfig: SIOCIFCREATE2: Invalid argument

Thanks.