Intel 82574L interface wedging - em7.3.2/8.2-STABLE
Jason Wolfe
nitroboost at gmail.com
Thu Mar 1 19:56:34 UTC 2012
So since the 7.3.0/7.3.2 code released out of the "Intel 82574L
interface wedging on em 7.1.9/7.2.3 when MSIX enabled" thread I've
been having some good results in 8.2-STABLE, and 'wedges' are much
less common. I am however still seeing them rarely, using some fuzzy
math based on uptime on the new code and number of boxes, about once
every 250 days. MUCH better than prior, but wondering if there is
something else still lingering? It appears to have the same symptoms
as before with a full buffer, where dropped packets start climbing and
packets out stall. These servers have MSI-X enabled.
I bounced em0 because dropped packets incremented 1036224 to 1036874
and the interface is not incrementing packets out.
9:15PM up 18 days, 5:07, 0 users, load averages: 1.75, 2.25, 2.35
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:25:90:2c:c3:a5
inet6 fe80::225:90ff:fe2c:c3a4%em0 prefixlen 64 scopeid 0x1
nd6 options=1<PERFORMNUD>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:25:90:2c:c3:a5
inet6 fe80::225:90ff:fe2c:c3a5%em1 prefixlen 64 scopeid 0x2
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=3<RXCSUM,TXCSUM>
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
inet 69.28.143.32 netmask 0xffffffff
inet 69.28.143.33 netmask 0xffffffff
inet 69.28.143.58 netmask 0xffffffff
inet 69.28.143.59 netmask 0xffffffff
inet 69.28.143.60 netmask 0xffffffff
inet 69.28.143.61 netmask 0xffffffff
inet 68.142.118.4 netmask 0xffffffff
inet 68.142.118.254 netmask 0xffffffff
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 00:25:90:2c:c3:a5
inet 69.164.10.184 netmask 0xffffff80 broadcast 69.164.10.255
inet6 fe80::225:90ff:fe2c:c3a5%lagg0 prefixlen 64 scopeid 0x5
inet6 2607:f4e8:320:14:225:90ff:fe2c:c3a5 prefixlen 64 autoconf
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
media: Ethernet autoselect
status: active
laggproto loadbalance
laggport: em0 flags=4<ACTIVE>
laggport: em1 flags=4<ACTIVE>
interrupt total rate
irq3: uart1 9150 0
cpu0: timer 3147281076 2000
irq256: em0:rx 0 92406843 58
irq257: em0:tx 0 7707454123 4897
irq258: em0:link 2 0
irq259: em1:rx 0 9059996943 5757
irq260: em1:tx 0 7697538938 4891
irq261: em1:link 10325 0
irq262: mps0 1109270744 704
cpu2: timer 3147265253 2000
cpu3: timer 3147265245 2000
cpu1: timer 3147265253 2000
Total 38255763895 24310
65436/10254/75690 mbufs in use (current/cache/total)
9092/4636/13728/5956826 mbuf clusters in use (current/cache/total/max)
9092/980 mbuf+clusters out of packet secondary zone in use (current/cache)
49150/877/50027/2978413 4k (page size) jumbo clusters in use
(current/cache/total/max)
0/0/0/6400 9k jumbo clusters in use (current/cache/total/max)
0/0/0/3200 16k jumbo clusters in use (current/cache/total/max)
231143K/15343K/246486K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0/0/0 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
195213 requests for I/O initiated by sendfile
0 calls to protocol drain routines
Name Mtu Network Address Ipkts Ierrs Idrop
Opkts Oerrs Coll Drop
em0 1500 <Link#1> 00:25:90:2c:c3:a5 91137540 0 0
49557533207 0 0 1039464
em0 1500 fe80::225:90f fe80::225:90ff:fe 0 - -
5 - - -
em1 1500 <Link#2> 00:25:90:2c:c3:a5 46770487879 52357 0
49003644601 0 0 132545
em1 1500 fe80::225:90f fe80::225:90ff:fe 0 - -
1 - - -
lagg0 1500 <Link#5> 00:25:90:2c:c3:a5 46861296630 0 0
98539892148 1172007 0 0
lagg0 1500 69.164.10.128 69.164.10.184 44434261867 - -
98536592195 - - -
lagg0 1500 fe80::225:90f fe80::225:90ff:fe 2737 - -
2750 - - -
lagg0 1500 2607:f4e8:320 2607:f4e8:320:14: 22188979 - -
22152621 - - -
kern.msgbuf:
Feb 27 21:15:07 cds447 kernel: Interface is RUNNING and ACTIVE
Feb 27 21:15:07 cds447 kernel: em0: hw tdh = 1385, hw tdt = 1385
Feb 27 21:15:07 cds447 kernel: em0: hw rdh = 452, hw rdt = 451
Feb 27 21:15:07 cds447 kernel: em0: Tx Queue Status = 0
Feb 27 21:15:07 cds447 kernel: em0: TX descriptors avail = 2048
Feb 27 21:15:07 cds447 kernel: em0: Tx Descriptors avail failure = 0
Feb 27 21:15:07 cds447 kernel: em0: RX discarded packets = 0
Feb 27 21:15:07 cds447 kernel: em0: RX Next to Check = 452
Feb 27 21:15:07 cds447 kernel: em0: RX Next to Refresh = 451
Feb 27 21:15:07 cds447 kernel: Interface is RUNNING and ACTIVE
Feb 27 21:15:07 cds447 kernel: em1: hw tdh = 221, hw tdt = 342
Feb 27 21:15:07 cds447 kernel: em1: hw rdh = 335, hw rdt = 233
Feb 27 21:15:07 cds447 kernel: em1: Tx Queue Status = 0
Feb 27 21:15:07 cds447 kernel: em1: TX descriptors avail = 2048
Feb 27 21:15:07 cds447 kernel: em1: Tx Descriptors avail failure = 0
Feb 27 21:15:07 cds447 kernel: em1: RX discarded packets = 0
Feb 27 21:15:07 cds447 kernel: em1: RX Next to Check = 832
Feb 27 21:15:07 cds447 kernel: em1: RX Next to Refresh = 838
net.inet.ip.intr_queue_maxlen: 512
net.inet.ip.intr_queue_drops: 0
dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.2
dev.em.0.%driver: em
dev.em.0.%location: slot=0 function=0
dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9
subdevice=0x10d3 class=0x020000
dev.em.0.%parent: pci1
dev.em.0.nvm: -1
dev.em.0.debug: -1
dev.em.0.fc: 3
dev.em.0.rx_int_delay: 0
dev.em.0.tx_int_delay: 66
dev.em.0.rx_abs_int_delay: 66
dev.em.0.tx_abs_int_delay: 66
dev.em.0.rx_processing_limit: 100
dev.em.0.eee_control: 0
dev.em.0.link_irq: 2
dev.em.0.mbuf_alloc_fail: 0
dev.em.0.cluster_alloc_fail: 0
dev.em.0.dropped: 0
dev.em.0.tx_dma_fail: 0
dev.em.0.rx_overruns: 0
dev.em.0.watchdog_timeouts: 0
dev.em.0.device_control: 1074790984
dev.em.0.rx_control: 67141634
dev.em.0.fc_high_water: 18432
dev.em.0.fc_low_water: 16932
dev.em.0.queue0.txd_head: 1385
dev.em.0.queue0.txd_tail: 1385
dev.em.0.queue0.tx_irq: 7707451554
dev.em.0.queue0.no_desc_avail: 0
dev.em.0.queue0.rxd_head: 659
dev.em.0.queue0.rxd_tail: 658
dev.em.0.queue0.rx_irq: 92406848
dev.em.0.mac_stats.excess_coll: 0
dev.em.0.mac_stats.single_coll: 0
dev.em.0.mac_stats.multiple_coll: 0
dev.em.0.mac_stats.late_coll: 0
dev.em.0.mac_stats.collision_count: 0
dev.em.0.mac_stats.symbol_errors: 0
dev.em.0.mac_stats.sequence_errors: 0
dev.em.0.mac_stats.defer_count: 0
dev.em.0.mac_stats.missed_packets: 0
dev.em.0.mac_stats.recv_no_buff: 0
dev.em.0.mac_stats.recv_undersize: 0
dev.em.0.mac_stats.recv_fragmented: 0
dev.em.0.mac_stats.recv_oversize: 0
dev.em.0.mac_stats.recv_jabber: 0
dev.em.0.mac_stats.recv_errs: 0
dev.em.0.mac_stats.crc_errs: 0
dev.em.0.mac_stats.alignment_errs: 0
dev.em.0.mac_stats.coll_ext_errs: 0
dev.em.0.mac_stats.xon_recvd: 0
dev.em.0.mac_stats.xon_txd: 0
dev.em.0.mac_stats.xoff_recvd: 0
dev.em.0.mac_stats.xoff_txd: 0
dev.em.0.mac_stats.total_pkts_recvd: 91202036
dev.em.0.mac_stats.good_pkts_recvd: 91202036
dev.em.0.mac_stats.bcast_pkts_recvd: 91083791
dev.em.0.mac_stats.mcast_pkts_recvd: 53524
dev.em.0.mac_stats.rx_frames_64: 91112331
dev.em.0.mac_stats.rx_frames_65_127: 26634
dev.em.0.mac_stats.rx_frames_128_255: 52891
dev.em.0.mac_stats.rx_frames_256_511: 6099
dev.em.0.mac_stats.rx_frames_512_1023: 4081
dev.em.0.mac_stats.rx_frames_1024_1522: 0
dev.em.0.mac_stats.good_octets_recvd: 5845701200
dev.em.0.mac_stats.good_octets_txd: 66047819213524
dev.em.0.mac_stats.total_pkts_txd: 49557613335
dev.em.0.mac_stats.good_pkts_txd: 49557613335
dev.em.0.mac_stats.bcast_pkts_txd: 35
dev.em.0.mac_stats.mcast_pkts_txd: 10502
dev.em.0.mac_stats.tx_frames_64: 191214392
dev.em.0.mac_stats.tx_frames_65_127: 5048362310
dev.em.0.mac_stats.tx_frames_128_255: 37903087
dev.em.0.mac_stats.tx_frames_256_511: 73859562
dev.em.0.mac_stats.tx_frames_512_1023: 491425065
dev.em.0.mac_stats.tx_frames_1024_1522: 43714848919
dev.em.0.mac_stats.tso_txd: 0
dev.em.0.mac_stats.tso_ctx_fail: 0
dev.em.0.interrupts.asserts: 3
dev.em.0.interrupts.rx_pkt_timer: 0
dev.em.0.interrupts.rx_abs_timer: 0
dev.em.0.interrupts.tx_pkt_timer: 0
dev.em.0.interrupts.tx_abs_timer: 0
dev.em.0.interrupts.tx_queue_empty: 0
dev.em.0.interrupts.tx_queue_min_thresh: 0
dev.em.0.interrupts.rx_desc_min_thresh: 0
dev.em.0.interrupts.rx_overrun: 0
dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.2
dev.em.1.%driver: em
dev.em.1.%location: slot=0 function=0
dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9
subdevice=0x10d3 class=0x020000
dev.em.1.%parent: pci2
dev.em.1.nvm: -1
dev.em.1.debug: -1
dev.em.1.fc: 3
dev.em.1.rx_int_delay: 0
dev.em.1.tx_int_delay: 66
dev.em.1.rx_abs_int_delay: 66
dev.em.1.tx_abs_int_delay: 66
dev.em.1.rx_processing_limit: 100
dev.em.1.eee_control: 0
dev.em.1.link_irq: 9701
dev.em.1.mbuf_alloc_fail: 0
dev.em.1.cluster_alloc_fail: 0
dev.em.1.dropped: 0
dev.em.1.tx_dma_fail: 0
dev.em.1.rx_overruns: 0
dev.em.1.watchdog_timeouts: 0
dev.em.1.device_control: 1074790984
dev.em.1.rx_control: 67141634
dev.em.1.fc_high_water: 18432
dev.em.1.fc_low_water: 16932
dev.em.1.queue0.txd_head: 428
dev.em.1.queue0.txd_tail: 592
dev.em.1.queue0.tx_irq: 7697553110
dev.em.1.queue0.no_desc_avail: 0
dev.em.1.queue0.rxd_head: 190
dev.em.1.queue0.rxd_tail: 189
dev.em.1.queue0.rx_irq: 8990697150
dev.em.1.mac_stats.excess_coll: 0
dev.em.1.mac_stats.single_coll: 0
dev.em.1.mac_stats.multiple_coll: 0
dev.em.1.mac_stats.late_coll: 0
dev.em.1.mac_stats.collision_count: 0
dev.em.1.mac_stats.symbol_errors: 0
dev.em.1.mac_stats.sequence_errors: 0
dev.em.1.mac_stats.defer_count: 0
dev.em.1.mac_stats.missed_packets: 52357
dev.em.1.mac_stats.recv_no_buff: 12234
dev.em.1.mac_stats.recv_undersize: 0
dev.em.1.mac_stats.recv_fragmented: 0
dev.em.1.mac_stats.recv_oversize: 0
dev.em.1.mac_stats.recv_jabber: 0
dev.em.1.mac_stats.recv_errs: 0
dev.em.1.mac_stats.crc_errs: 0
dev.em.1.mac_stats.alignment_errs: 0
dev.em.1.mac_stats.coll_ext_errs: 0
dev.em.1.mac_stats.xon_recvd: 0
dev.em.1.mac_stats.xon_txd: 0
dev.em.1.mac_stats.xoff_recvd: 0
dev.em.1.mac_stats.xoff_txd: 0
dev.em.1.mac_stats.total_pkts_recvd: 46770604210
dev.em.1.mac_stats.good_pkts_recvd: 46770551853
dev.em.1.mac_stats.bcast_pkts_recvd: 91073541
dev.em.1.mac_stats.mcast_pkts_recvd: 53519
dev.em.1.mac_stats.rx_frames_64: 14339105496
dev.em.1.mac_stats.rx_frames_65_127: 23186724273
dev.em.1.mac_stats.rx_frames_128_255: 38625485
dev.em.1.mac_stats.rx_frames_256_511: 116531779
dev.em.1.mac_stats.rx_frames_512_1023: 165521693
dev.em.1.mac_stats.rx_frames_1024_1522: 8924043127
dev.em.1.mac_stats.good_octets_recvd: 16290987983606
dev.em.1.mac_stats.good_octets_txd: 65599822022242
dev.em.1.mac_stats.total_pkts_txd: 49003789456
dev.em.1.mac_stats.good_pkts_txd: 49003789452
dev.em.1.mac_stats.bcast_pkts_txd: 10085
dev.em.1.mac_stats.mcast_pkts_txd: 11
dev.em.1.mac_stats.tx_frames_64: 170250212
dev.em.1.mac_stats.tx_frames_65_127: 4806984601
dev.em.1.mac_stats.tx_frames_128_255: 36525028
dev.em.1.mac_stats.tx_frames_256_511: 71876326
dev.em.1.mac_stats.tx_frames_512_1023: 482360156
dev.em.1.mac_stats.tx_frames_1024_1522: 43435793133
dev.em.1.mac_stats.tso_txd: 0
dev.em.1.mac_stats.tso_ctx_fail: 0
dev.em.1.interrupts.asserts: 7943
dev.em.1.interrupts.rx_pkt_timer: 1
dev.em.1.interrupts.rx_abs_timer: 0
dev.em.1.interrupts.tx_pkt_timer: 0
dev.em.1.interrupts.tx_abs_timer: 1
dev.em.1.interrupts.tx_queue_empty: 0
dev.em.1.interrupts.tx_queue_min_thresh: 0
dev.em.1.interrupts.rx_desc_min_thresh: 0
dev.em.1.interrupts.rx_overrun: 6
hw.em.eee_setting: 0
hw.em.rx_process_limit: 100
hw.em.enable_msix: 1
hw.em.sbp: 0
hw.em.smart_pwr_down: 0
hw.em.txd: 2048
hw.em.rxd: 2048
hw.em.rx_abs_int_delay: 66
hw.em.tx_abs_int_delay: 66
hw.em.rx_int_delay: 0
hw.em.tx_int_delay: 66
Jason
More information about the freebsd-net
mailing list