From nobody Mon Nov 14 20:50:57 2022 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NB1g55XNbz4dKy7 for ; Mon, 14 Nov 2022 20:50:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NB1g546n0z3QH2 for ; Mon, 14 Nov 2022 20:50:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1668459057; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Kr2ZGkXA90GuS4ZKdXS4jVhA94g1BuouAdsX4PuBgmU=; b=X+1TQAlUQWqZRdmKaBOfYZeHt8Y4JTt5zNaReQm5R2w/MvTjZFCfPf0AEW9qTArjEyHn2P HWHbD6zrm4/IDOGHT+j8gLzukun1AEY5cU/5YuBSReS9S9hq2rY8a028w5WW78Lb84GXT9 dpUdH2v6bt5hOhZYpBu++HcgJxpIfDa5I3ya0aGJcwAgX+BJRPdOMZyyyvK7vTf+Rw2SJk L1bcvdivtLgrc7K7m/4/9EhWAFAhiDxUZx15h1oC33Oua7f7gIXM5ijZfWdty8xoSSbfjU B4+cuMaKQSeQDTRmfVnmhzjIBQjFeiSH7fkHxWjbOc1Siup80GkYJkaV/Ntf4w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1668459057; a=rsa-sha256; cv=none; b=oH1yLbVJP++a6I8ajGztpehJpZ9YTEeVYphiZVMBPyzp+M4igpCSo0wJuQZCp9iZWLZFKY 8YBkCPJs0FRGOO/6/2pqGcLo7LpBzuI3uxgwwekWuAYyKI4GvqCILbOnkOu8Patu++zKDc OrHIK6bYuYPH12cnja5QF2CTqc8qTFLxm+nm2oZfOtYOHHA71wv33Ukh41jFBzJcMyVaw+ LGMduBV2QuYNGGUpkdTV0IcEdWz6mYvEJq/LU6futsUvKwAyQXT120bRHgvZjasGeQPve4 iy9Gb67mE2Fo9VCHDLrR67hQMenXz4BbTg095nMEpHOI/V4xav2RhkKP60nqLw== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4NB1g53Bfyzvnr for ; Mon, 14 Nov 2022 20:50:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 2AEKovYG043594 for ; Mon, 14 Nov 2022 20:50:57 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 2AEKov4c043593 for virtualization@FreeBSD.org; Mon, 14 Nov 2022 20:50:57 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 267769] Bhyve core dump on suspend/resume of virtio-scsi device Date: Mon, 14 Nov 2022 20:50:57 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: dan@sunsaturn.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D267769 Bug ID: 267769 Summary: Bhyve core dump on suspend/resume of virtio-scsi device Product: Base System Version: CURRENT Hardware: amd64 OS: Any Status: New Severity: Affects Many People Priority: --- Component: bhyve Assignee: virtualization@FreeBSD.org Reporter: dan@sunsaturn.com Bhyve core dumps after using virtio-scsi device with suspend/resume with ex= it code 139. After a resume of guest, a simple "df" or any other command will freeze and within 3-5 min the bhyve process will dump core. This seems to o= nly affect virtio-scsi , when I tested virtio-blk all seems well and guest resu= mes fine. router:/root/guests # uname -a FreeBSD router.sunsaturn.com 14.0-CURRENT FreeBSD 14.0-CURRENT #1 main-n259058-105019e0d6c: Sat Nov 5 05:37:28 CDT 2022=20=20=20=20 dan@router.sunsaturn.com:/usr/obj/usr/src/amd64.amd64/sys/MYKERNEL amd64 router:/root/guests #=20 Steps to reproduce: ######################################/etc/ctl.conf=20 portal-group pg0 { discovery-auth-group no-authentication listen 127.0.0.1:3260 } #asterisk target iqn.com.sunsaturn.asterisk:target1 { auth-group no-authentication portal-group pg0 #bhyve virti-iscsi disk - /dev/cam/ctl1.0 port ioctl/1 lun 0 { path /dev/zvol/zroot/asterisk=20 #blocksize 128 serial 000c2937247001 device-id "iSCSI Disk 000c2937247001" option vendor "FreeBSD" option product "iSCSI Disk" option revision "0123" option insecure_tpc on } lun 1 { path /vm/.iso/FreeBSD-14.0-CURRENT-amd64-20221103-5cc5c9254da-259005-disc1.iso #byhve seems to just hang when I set it to an actual CDROM = so let it default to type 0 #device-type 5 serial 000c2937247003 device-id "iSCSI CDROM ISO 000c2937247002" option vendor "FreeBSD CDROM" option product "iSCSI CDROM" option revision "0123" option insecure_tpc on } } ###########################/etc/iscsi.conf #asterisk t1 { TargetAddress =3D 127.0.0.1:3260 TargetName =3D iqn.com.sunsaturn.asterisk:target1 } ############################/etc/rc.conf########## ctld_enable=3D"YES" #load /etc/ctl.conf iscsid_enable=3D"YES" #start iscsid process to connect to ctld #client - service iscsictl start iscsictl_enable=3D"YES" #connect to all targets in /etc/iscsi.conf iscsictl_flags=3D"-Aa" ##################asterisk.sh test script################### #!/bin/bash # # General script to test bhyve suspend/resume features # # Requirements: FreeBSD current # screen # git clone https://git.FreeBSD.org/src.git /usr/src # cd /usr/src/sys/amd64/conf (edit MYKERNEL) cp GENERIC MYKERNEL-NODEBUG; (= add: options BHYVE_SNAPSHOT) # cd /usr/src # (find amount of CPUs and adjust -j below - "dmesg|grep SMP") # make -j12 buildworld -DWITH_BHYVE_SNAPSHOT -DWITH_MALLOC_PRODUCTION # make -j12 buildkernel KERNCONF=3DMYKERNEL # make installkernel KERNCONF=3DMYKERNEL # shutdown -r now # cd /usr/src; make installworld # shutdown -r now # etcupdate -B # pkg bootstrap -f #if new freebsd version # pkg upgrade -f #if new freebsd version=20 # # Report anomolies to dan@sunsaturn.com ##############EDIT ME##################### HOST=3D"127.0.0.1" # vncviewer 127.0.0.1:5900 - pkg install tightvnc PORT=3D"5900" WIDTH=3D"800" HEIGHT=3D"600" VMNAME=3D"asterisk" ISO=3D"/vm/.iso/FreeBSD-14.0-CURRENT-amd64-20221103-5cc5c9254da-259005-disc= 1.iso" DIR=3D"/vm/asterisk" # Used to hold files when guest suspended SERIAL=3D"/dev/nmdm_asteriskA" # For "screen /dev/nmdm_asteriskB"= - pkg install screen TAP=3D"tap0" CPU=3D"8" RAM=3D"8G" #For testing virtio-scsi STORAGE=3D"/dev/cam/ctl1.0" # port from /etc/ctl.conf(port io= ctl/1) - core dumping on resume DEVICE=3D"virtio-scsi" #for testing virtio-blk # Comment out above 2 lines if using these #DEVICE=3D"virtio-blk" #STORAGE=3D"/dev/zvol/zroot/asterisk" # Standard zvol #STORAGE=3D"/dev/da1" # Block device created from iscsi= ctl ######################################### usage() { echo "Usage: $1 start (Start the guest: $VMNAME)";=20 echo "Usage: $1 stop (Stop the guest: $VMNAME)";=20 echo "Usage: $1 resume (Resume the guest from last suspend: $VMNAME)";= =20 echo "Usage: $1 suspend (Suspend the guest: $VMNAME)";=20 echo "Usage: $1 install (Install new guest: $VMNAME)";=20 exit } if [ ! -d "$DIR" ]; then=20 mkdir -p $DIR fi #if [ -z "$2" ]; then # usage #else # VMNAME=3D$2 #fi if [ "$1" =3D=3D "install" ]; then #Kill it before starting it echo "Execute: screen $SERIAL" bhyvectl --destroy --vm=3D$VMNAME bhyve -c $CPU -m $RAM -w -H -A \ -s 0:0,hostbridge \ -s 3:0,ahci-cd,$ISO \ -s 4:0,$DEVICE,$STORAGE \ -s 5:0,virtio-net,$TAP \ -s 29,fbuf,tcp=3D$HOST:$PORT,w=3D$WIDTH,h=3D$HEIGHT \ -s 30,xhci,tablet \ -s 31,lpc -l com1,stdio \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ $VMNAME #kill it after=20 bhyvectl --destroy --vm=3D$VMNAME elif [ "$1" =3D=3D "start" ]; then=20 while true do echo "Starting $VMNAME -s 29,fbuf,tcp=3D$HOST:$PORT,w=3D$WIDTH,h=3D$H= EIGHT" #Kill it before starting it bhyvectl --destroy --vm=3D$VMNAME > /dev/null 2>&1 bhyve -c $CPU -m $RAM -w -H -A \ -s 0:0,hostbridge \ -s 4:0,$DEVICE,$STORAGE \ -s 5:0,virtio-net,$TAP \ -s 29,fbuf,tcp=3D$HOST:$PORT,w=3D$WIDTH,h=3D$HEIGHT \ -s 30,xhci,tablet \ -s 31,lpc -l com1,$SERIAL \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ $VMNAME #DISABLING REBOOT LOOP AS SUSPEND RETURNS ERROR CODE 0 AS WELL #if [ "$?" !=3D 0 ]; #then # echo "The exit code was not reboot code 0!: $?" # exit #fi echo "The exit code was : $?" exit done elif [ "$1" =3D=3D "resume" ]; then=20 while true do echo "Starting $VMNAME -s 29,fbuf,tcp=3D$HOST:$PORT,w=3D$WIDTH,h=3D$H= EIGHT" #Kill it before starting it bhyvectl --destroy --vm=3D$VMNAME > /dev/null 2>&1 if [ -f "$DIR/default.ckp" ]; then bhyve -c $CPU -m $RAM -w -H -A \ -s 0:0,hostbridge \ -s 4:0,$DEVICE,$STORAGE \ -s 5:0,virtio-net,$TAP \ -s 29,fbuf,tcp=3D$HOST:$PORT,w=3D$WIDTH,h=3D$HEIGHT \ -s 30,xhci,tablet \ -s 31,lpc -l com1,$SERIAL \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ -r $DIR/default.ckp \ $VMNAME else echo "Guest was never suspended" exit fi #DISABLING REBOOT LOOP AS SUSPEND RETURNS ERROR CODE 0 AS WELL #if [ "$?" !=3D 0 ]; #then # echo "The exit code was not reboot code 0!: $?" # exit #fi echo "The exit code was : $?" exit done elif [ "$1" =3D=3D "suspend" ]; then=20 bhyvectl --suspend $DIR/default.ckp --vm=3D$VMNAME elif [ "$1" =3D=3D "stop" ]; then=20 bhyvectl --destroy --vm=3D$VMNAME=20 else=20 usage fi ##################Making it core dump###################### #terminal 1 router:/root/guests # ./asterisk.sh start Starting asterisk -s 29,fbuf,tcp=3D127.0.0.1:5900,w=3D800,h=3D600 fbuf frame buffer base: 0x2275eb000000 [sz 16777216] Unhandled ps2 mouse command 0xe1 #terminal 2 router:/root/guests # ./asterisk.sh suspend router:/root/guests # ./asterisk.sh resume Starting asterisk -s 29,fbuf,tcp=3D127.0.0.1:5900,w=3D800,h=3D600 fbuf frame buffer base: 0x1b6423400000 [sz 16777216] Pausing pci devs... pci_pause: no such name: virtio-blk pci_pause: no such name: ahci pci_pause: no such name: ahci-hd pci_pause: no such name: ahci-cd Restoring vm mem... [8192.000MiB / 8192.000MiB] |##########################################################################= ###################################################################| Restoring pci devs... vm_restore_user_dev: Device size is 0. Assuming virtio-blk is not used vm_restore_user_dev: Device size is 0. Assuming virtio-rnd is not used vm_restore_user_dev: Device size is 0. Assuming e1000 is not used vm_restore_user_dev: Device size is 0. Assuming ahci is not used vm_restore_user_dev: Device size is 0. Assuming ahci-hd is not used vm_restore_user_dev: Device size is 0. Assuming ahci-cd is not used Restoring kernel structs... Resuming pci devs... pci_resume: no such name: virtio-blk pci_resume: no such name: ahci pci_resume: no such name: ahci-hd pci_resume: no such name: ahci-cd ##########at this point just go to guest and type "df" or any command and w= atch it hang, wait a few minutes, then go back to Terminal 2: ./asterisk.sh: line 147: 34302 Segmentation fault (core dumped) bhyve = -c $CPU -m $RAM -w -H -A -s 0:0,hostbridge -s 4:0,$DEVICE,$STORAGE -s 5:0,virtio-net,$TAP -s 29,fbuf,tcp=3D$HOST:$PORT,w=3D$WIDTH,h=3D$HEIGHT -s 30,xhci,tablet -s 31,lpc -l com1,$SERIAL -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -r $DIR/default.ckp $VMNAME The exit code was : 139 router:/root/guests # --=20 You are receiving this mail because: You are the assignee for the bug.=