[Bug 273372] SR-IOV Networking in Bhyve Causes Chelsio T520-SO-CR to Fail on Host, Kernel Panic if Reset

From: <bugzilla-noreply_at_freebsd.org>
Date: Sat, 09 Sep 2023 19:51:38 UTC
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273372

--- Comment #4 from Mark McBride <mark@markmcb.com> ---
I just attempted 14 vfs with 14-BETA1 as the host. Still fails. As requested
...

# sysctl -n dev.t5nex.0.misc.devlog                                             
      Seq#           Tstamp     Level  Facility  Message                        
         0           427889      INFO      CORE  log initialized @ 0x200a0000
size 32768 (128 entries) fwrev 0x011b0400 pcie_fw 0x0014cc10                    
         1           521576      INFO      CORE  bootstrap firmware took 27
msecs to run                                                                    
         2           525170      INFO      CORE  Serial Configuration version:
0x1006000 VPD version: 0x1                                                      
         3           525173      INFO      CORE  pcie: npf 7 (pfbitmap 0x7f)
nvf 64 (pf 0..7 0x1010101000000000) vfstride 4                                  
         4           535983      INFO      CORE  flr_timer_start: flowc_id 125
0x205fff80 buf 0x205fc0c0                                                       
         5           575529      INFO        CF  cf_parse: file memtype 0x1
memaddr 0x5e0000 mapped @ 0x205e0000:                                           
         6           575559      INFO        CF  configuration file parser: pl
timeout value is too large, changing from 10000 to 4194usecs                    
         7           576541      INFO      CORE  configured with caps nbm|link
0x00000000 switch|nic 0x00030001 toe|rdma 0x00010003 iscsi|crypto 0x00410000
fcoe:0x0                                                          
         8           576579    NOTICE      CORE  pfvf_init: 150 SMT entires are
still unused                                                                    
         9           576592      INFO        HW  hw_tp_tcp_tunings: tuning for
LAN environment                                                                 
        10           576622      INFO       RES  le configuration: nentries
2048 route 32 clip 32 filter 1440 server 416 active 128 hash 0 nserversram 0    
        11           577565      INFO      DMAQ  Firmware using new VIID format
([11:9] PFN, [8] VI Valid, [7:0] VIN)                                           
        12           578623      INFO       RES  flowc_table 0x200c8000, size
211456                                                                          
        13           580825    NOTICE        RI  Not enough memory for 21
ird/ord per conn, ird/ord changed to 21                                         
        14           582460      INFO      CORE  flr_timer_start: flowc_id 1651
0x200fb980 buf 0x2039c800                                                       
        15           612719      INFO       RES  le initialization: nentries
2048 route 32 clip 32 filter 1440 server 416 active 128 hash 0 nserversram 0    
        16           836644      INFO      PORT  module[0]: gpio 11 vendor id
004020, identifier 0x03, SFP28(byte 36/192) 0xff, SFP(byte 3/131) 0x00, 1G
(byte 6) 0x00                                                        
        17           836646      INFO      PORT  optical length(byte 15/142) 0,
copper cable(byte 8/147) 0x04, length(byte 18/146) 3, module_type 0x04          
        18           836649      INFO      PORT  hw_mac_init_port[0], ptype
0x9, speed 0x4, lanes 0x1, fec 0x0                                              
        19           836685      INFO        TM  pktsched channel 0 sets speed
(from 0) to 10000000 kbps                                                       
        20           836693      INFO      PORT  port_init: TRY_OTHER_SPEED
flag is default set for T5                                                      
        21           836696      INFO      PORT  port_refill_other_speed:
module speed 0x4, try speed 0x6                                                 
        22           842475      INFO      PORT  module[1]: gpio 15 vendor id
004020, identifier 0x03, SFP28(byte 36/192) 0xff, SFP(byte 3/131) 0x00, 1G
(byte 6) 0x00                                                        
        23           842476      INFO      PORT  optical length(byte 15/142) 0,
copper cable(byte 8/147) 0x04, length(byte 18/146) 3, module_type 0x04          
        24           842478      INFO      PORT  hw_mac_init_port[1], ptype
0x9, speed 0x4, lanes 0x2, fec 0x0                                              
        25           842503      INFO        TM  pktsched channel 1 sets speed
(from 0) to 10000000 kbps                                                       
        26           842509      INFO      PORT  port_init: TRY_OTHER_SPEED
flag is default set for T5                                                      
        27           842510      INFO      PORT  port_refill_other_speed:
module speed 0x4, try speed 0x6                                                 
        28           842521      INFO        TM  pktsched channel 2 sets speed
(from 0) to 1000000 kbps                                                        
        29           842534      INFO        TM  pktsched channel 3 sets speed
(from 0) to 1000000 kbps                                                        
        30         11621579    NOTICE       FLR  pfn 1 vfn 1 via command        
        31         11675139    NOTICE       FLR  pfn 1 vfn 2 via command        
        32         11732187    NOTICE       FLR  pfn 1 vfn 3 via command        
        33         11784054    NOTICE       FLR  pfn 1 vfn 4 via command        
        34         11836035    NOTICE       FLR  pfn 1 vfn 5 via command
        35         11888064    NOTICE       FLR  pfn 1 vfn 6 via command
        36         11947325    NOTICE       FLR  pfn 1 vfn 7 via command
        37         12002982    NOTICE       FLR  pfn 1 vfn 8 via command
        38         12061187    NOTICE       FLR  pfn 1 vfn 9 via command
        39         12114554    NOTICE       FLR  pfn 1 vfn 10 via command
        40         12442526      INFO      PORT  port_link_state_handler[0]
powering up
        41         12442541      INFO      PORT  port[0] update (flowcid 1456
rc 0)
        42         12742529      INFO      PORT  port_hss_sigdet[0]: hss_sigdet
changed to 0x1
        43         12742533      INFO      PORT  hw_mac_link_status[0]
int_cause 0x13019f0, link_status 0x12
        44         12942526      INFO      PORT  port[0] link up (1) (speed 0x4
acaps 0x70004 lpcaps 0x70000)
        45         12942531      INFO      PORT  port[0] set PAUSE PARAMS:
pppen 0 txpe 0x1 rxpe 0x1
        46         12942539      INFO      PORT  port[0] update (flowcid 1456
rc 0)
        47         26080128       ERR      CORE  insufficient caps to process
mailbox cmd: pfn 0x1 vfn 0x1; r_caps 0x86 wx_caps 0x82 required r_caps 0x81
w_caps 0x5
        48         26142587      INFO      PORT  port_link_state_handler[1]
powering up
        49         26142593      INFO      PORT  port[1] update (flowcid 1475
rc 0)
        50         26442579      INFO      PORT  port_hss_sigdet[1]: hss_sigdet
changed to 0x2
        51         28492130       ERR      CORE  insufficient caps to process
mailbox cmd: pfn 0x1 vfn 0x2; r_caps 0x86 wx_caps 0x82 required r_caps 0x81
w_caps 0x5
        52         28542596      INFO      PORT  port[1] update (flowcid 1475
rc 0)
        53         28542600      INFO      PORT  port[1] update (flowcid 1476
rc 0)
        54         30042592      INFO      PORT  port_hss_sigdet[1]: hss_sigdet
changed to 0x0
        55         30642594      INFO      PORT  port_hss_sigdet[1]: hss_sigdet
changed to 0x2
        56         30642596      INFO      PORT  hw_mac_link_status[1]
int_cause 0x10018f0, link_status 0x12
        57         30742595      INFO      PORT  hw_mac_link_status[1]
int_cause 0x10018f0, link_status 0x12
        58         30942601      INFO      PORT  port[1] link up (1) (speed 0x4
acaps 0x70004 lpcaps 0x70000)
        59         30942605      INFO      PORT  port[1] set PAUSE PARAMS:
pppen 0 txpe 0x1 rxpe 0x1
        60         30942614      INFO      PORT  port[1] update (flowcid 1475
rc 0)
        61         30942617      INFO      PORT  port[1] update (flowcid 1476
rc 0)
        62         30948649       ERR      CORE  insufficient caps to process
mailbox cmd: pfn 0x1 vfn 0x3; r_caps 0x86 wx_caps 0x82 required r_caps 0x81
w_caps 0x5
        63         31042603      INFO      PORT  port[1] update (flowcid 1475
rc 0)
        64         31042607      INFO      PORT  port[1] update (flowcid 1476
rc 0)
        65         31042610      INFO      PORT  port[1] update (flowcid 1477
rc 0)
        66         33283779       ERR      CORE  insufficient caps to process
mailbox cmd: pfn 0x1 vfn 0x4; r_caps 0x86 wx_caps 0x82 required r_caps 0x81
w_caps 0x5
        67         33342616      INFO      PORT  port[1] update (flowcid 1475
rc 0)
        68         33342621      INFO      PORT  port[1] update (flowcid 1476
rc 0)
        69         33342625      INFO      PORT  port[1] update (flowcid 1477
rc 0)
        70         33342628      INFO      PORT  port[1] update (flowcid 1478
rc 0)
        71         36739130       ERR      CORE  insufficient caps to process
mailbox cmd: pfn 0x1 vfn 0x6; r_caps 0x86 wx_caps 0x82 required r_caps 0x81
w_caps 0x5
        72         36742629      INFO      PORT  port[1] update (flowcid 1475
rc 0)
        73         36742634      INFO      PORT  port[1] update (flowcid 1476
rc 0)
        74         36742637      INFO      PORT  port[1] update (flowcid 1477
rc 0)
        75         36742641      INFO      PORT  port[1] update (flowcid 1478
rc 0)
        76         36742644      INFO      PORT  port[1] update (flowcid 1480
rc 0)
        77         39149129       ERR      CORE  insufficient caps to process
mailbox cmd: pfn 0x1 vfn 0x7; r_caps 0x86 wx_caps 0x82 required r_caps 0x81
w_caps 0x5
        78         39242638      INFO      PORT  port[1] update (flowcid 1475
rc 0)
        79         39242643      INFO      PORT  port[1] update (flowcid 1476
rc 0)
        80         39242646      INFO      PORT  port[1] update (flowcid 1477
rc 0)
        81         39242650      INFO      PORT  port[1] update (flowcid 1478
rc 0)
        82         39242653      INFO      PORT  port[1] update (flowcid 1480
rc 0)
        83         39242656      INFO      PORT  port[1] update (flowcid 1481
rc 0)
        84         43102156       ERR      CORE  insufficient caps to process
mailbox cmd: pfn 0x1 vfn 0x8; r_caps 0x86 wx_caps 0x82 required r_caps 0x81
w_caps 0x5
        85         43142655      INFO      PORT  port[1] update (flowcid 1475
rc 0)
        86         43142659      INFO      PORT  port[1] update (flowcid 1476
rc 0)
        87         43142663      INFO      PORT  port[1] update (flowcid 1477
rc 0)
        88         43142666      INFO      PORT  port[1] update (flowcid 1478
rc 0)
        89         43142670      INFO      PORT  port[1] update (flowcid 1480
rc 0)
        90         43142673      INFO      PORT  port[1] update (flowcid 1481
rc 0)
        91         43142676      INFO      PORT  port[1] update (flowcid 1482
rc 0)
        92       1548458671      INFO       FLR  pfn 1 vfn 11 FSM start
        93       1548528664      INFO       FLR  pfn 1 vfn 11 FSM complete
        94       1600183614    NOTICE       FLR  pfn 1 vfn 11 via command
        95       1643893645       ERR      CORE  insufficient caps to process
mailbox cmd: pfn 0x1 vfn 0xb; r_caps 0x86 wx_caps 0x82 required r_caps 0x81
w_caps 0x5
        96       1643949059      INFO      PORT  port[1] update (flowcid 1475
rc 0)
        97       1643949064      INFO      PORT  port[1] update (flowcid 1476
rc 0)
        98       1643949068      INFO      PORT  port[1] update (flowcid 1477
rc 0)
        99       1643949072      INFO      PORT  port[1] update (flowcid 1478
rc 0)
       100       1643949075      INFO      PORT  port[1] update (flowcid 1480
rc 0)
       101       1643949078      INFO      PORT  port[1] update (flowcid 1481
rc 0)
       102       1643949081      INFO      PORT  port[1] update (flowcid 1482
rc 0)
       103       1643949084      INFO      PORT  port[1] update (flowcid 1485
rc 0)

-- 
You are receiving this mail because:
You are the assignee for the bug.