netstat -B "Recv"
elof2 at sentor.se
elof2 at sentor.se
Thu Nov 5 11:39:56 UTC 2015
Hi Christian.
Sorry, I was in a bit of a hurry when I wrote the below.
Here's more information:
FreeBSD 10.1 amd64 (on both boxes).
In rc.conf on the receiver I have:
ifconfig_ix0="up -arp -lro"
ifconfig_ix1="up -arp -lro"
cloned_interfaces="bridge0"
ifconfig_bridge0="up addm ix0 -discover ix0 -learn ix0 private ix0 \
addm ix1 -discover ix1 -learn ix1 private ix1 \
maxaddr 1 -arp monitor name mon0"
So 'mon0' is a monitor interface consisting of ix0 and ix1.
During my test, ix0 has no carrier, only ix1 is in use.
On the sending box I run tcpreplay to send 2000000 packets for each run.
Naturally I run the netstat commands on the receiving machine after the
tcpreplay on the other machine has finished, not during. :-)
I reboot to start over from scratch.
I start a sniffer:
# tcpdump -nli mon0 -w /dev/null
# netstat -in
Name Mtu Network Address Ipkts Ierrs Idrop Opkts
Oerrs Coll
ix0 1500 <Link#1> 0c:c4:7a:58:e2:3c 0 0 0 0
0 0
ix1 1500 <Link#2> 0c:c4:7a:58:e2:3d 0 0 0 0
0 0
mon0 1500 <Link#8> 02:74:e9:56:9c:00 0 0 0 0
0 0
# netstat -B
Pid Netif Flags Recv Drop Match Sblen Hblen Command
1422 mon0 p--s--- 0 0 0 0 0 tcpdump
I send 2000000 packets.
The receiver now show:
# netstat -in
Name Mtu Network Address Ipkts Ierrs Idrop Opkts
Oerrs Coll
ix0 1500 <Link#1> 0c:c4:7a:58:e2:3c 0 0 0 0
0 0
ix1 1500 <Link#2> 0c:c4:7a:58:e2:3d 2000000 0 0 0
0 0
mon0 1500 <Link#8> 02:74:e9:56:9c:00 2000000 0 0 0
0 0
# netstat -B
Pid Netif Flags Recv Drop Match Sblen Hblen Command
1422 mon0 p--s--- 1999851 0 2000000 0 0 tcpdump
I press ctrl-c in tcpdump and it says:
2000000 packets captured
1999851 packets received by filter
0 packets dropped by kernel
So the BPF stats from both netstat and tcpdump look quite strange. :-)
I start a new tcpdump and send 2000000 new packets.
I now run 'netstat -B' a few times during the transfer, and one can see
that the diffing packets are evenly distributed over time:
Pid Netif Flags Recv Drop Match Sblen Hblen Command
1557 mon0 p--s--- 0 0 0 0 0 tcpdump
1557 mon0 p--s--- 383905 0 383923 1993540 0 tcpdump
1557 mon0 p--s--- 651010 0 651038 1112814 0 tcpdump
1557 mon0 p--s--- 944090 0 944128 2883633 0 tcpdump
1557 mon0 p--s--- 1125245 0 1125292 2034033 0 tcpdump
1557 mon0 p--s--- 1319456 0 1319514 467992 0 tcpdump
1557 mon0 p--s--- 1516545 0 1516618 4183543 0 tcpdump
1557 mon0 p--s--- 1760527 0 1760608 1365257 0 tcpdump
1557 mon0 p--s--- 1999908 0 2000000 2870940 0 tcpdump
1557 mon0 p--s--- 1999908 0 2000000 0 0 tcpdump
1557 mon0 p--s--- 1999908 0 2000000 0 0 tcpdump
ctrl-c on tcpdump says:
2000000 packets captured
1999908 packets received by filter
0 packets dropped by kernel
I run a tcpdump directly on ix1, instead of the bridge, and try again.
Same thing, so the bridge has nothing to do with it:
Pid Netif Flags Recv Drop Match Sblen Hblen Command
1819 ix1 p--s--- 1999904 0 2000000 0 0 tcpdump
^C
2000000 packets captured
1999904 packets received by filter
0 packets dropped by kernel
I rebooted it again, started tcpdump on ix1 and received 2000000 packets.
By accident I forgot to kill other processes that might have an impact on
my tests. Maybe that was a good thing, because 'netstat -B' now show:
Pid Netif Flags Recv Drop Match Sblen Hblen Command
921 mon0 p--s--- 2000000 76390 2000000 0 0 suricata
1358 ix1 p--s--- 1999897 0 2000000 0 0 tcpdump
So the BPF stats for suricata is showing 2000000 received and 2000000
matched (and a few drops) as it should.
Just to see if there would be any difference, I tried running tcpdump
linebuffered but throwing away the output:
# tcpdump -nli ix1 > /dev/null
# netstat -B
Pid Netif Flags Recv Drop Match Sblen Hblen Command
921 mon0 p--s--- 3999997 214356 4000000 0 0 suricata
1521 ix1 p--s--- 1999875 46200 2000000 0 0 tcpdump
^C
1953800 packets captured
1999875 packets received by filter
46200 packets dropped by kernel
Nope, same-same. (I don't know what I was expecting)
Since tcpdump is the one showing odd values I test with tshark
(dumpcap) instead:
Pid Netif Flags Recv Drop Match Sblen Hblen Command
921 mon0 p--s--- 5999996 473528 6000000 0 0 suricata
1624 ix1 p--s--- 1999920 0 2000000 0 0 dumpcap
Nope same problem there. Tshark received 2000000 packets but for some
reason, the BPF stats show 1999920 received packets.
Also, this run suricata also had the same problem, so maybe it was just a
coincidence that it showed correct values that first run.
I try again, this time with ngrep as the sniffer:
# ngrep -d ix1 > /dev/null
Pid Netif Flags Recv Drop Match Sblen Hblen Command
921 mon0 p--s--- 7999950 766788 8000000 0 0 suricata
1730 ix1 p--s--- 1999862 0 0 0 0 ngrep
Ngrep also show too few recv packets (and zero matches for some reason).
Suricata recv-stats had problems this run as well, so it must have been a
fluke that it got correct values above.
Here's some ix stats, in case they should reveal something interesting:
# netstat -in
Name Mtu Network Address Ipkts Ierrs Idrop Opkts
Oerrs Coll
ix0 1500 <Link#1> 0c:c4:7a:58:e2:3c 0 0 0 0
0 0
ix1 1500 <Link#2> 0c:c4:7a:58:e2:3d 12000000 0 0 0
0 0
mon0 1500 <Link#8> 02:74:e9:56:9c:00 11999998 0 0 0
0 0
# sysctl -a | egrep "ix\.|ixgbe" | grep -v "ix\.0" | more
device ixgbe
hw.ix.enable_aim: 1
hw.ix.max_interrupt_rate: 31250
hw.ix.rx_process_limit: 256
hw.ix.tx_process_limit: 256
hw.ix.enable_msix: 1
hw.ix.num_queues: 8
hw.ix.txd: 2048
hw.ix.rxd: 2048
dev.ix.%parent:
dev.ix.1.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version -
2.5.15
dev.ix.1.%driver: ix
dev.ix.1.%location: slot=0 function=1
dev.ix.1.%pnpinfo: vendor=0x8086 device=0x1528 subvendor=0x15d9
subdevice=0x0734 class=0x020000
dev.ix.1.%parent: pci1
dev.ix.1.fc: 3
dev.ix.1.enable_aim: 1
dev.ix.1.advertise_speed: 0
dev.ix.1.ts: 0
dev.ix.1.dropped: 0
dev.ix.1.mbuf_defrag_failed: 0
dev.ix.1.watchdog_events: 0
dev.ix.1.link_irq: 2
dev.ix.1.queue0.interrupt_rate: 500000
dev.ix.1.queue0.irqs: 1500836
dev.ix.1.queue0.txd_head: 0
dev.ix.1.queue0.txd_tail: 0
dev.ix.1.queue0.tso_tx: 0
dev.ix.1.queue0.no_tx_dma_setup: 0
dev.ix.1.queue0.no_desc_avail: 0
dev.ix.1.queue0.tx_packets: 0
dev.ix.1.queue0.rxd_head: 1704
dev.ix.1.queue0.rxd_tail: 1703
dev.ix.1.queue0.rx_packets: 1500840
dev.ix.1.queue0.rx_bytes: 676084080
dev.ix.1.queue0.rx_copies: 959400
dev.ix.1.queue0.lro_queued: 0
dev.ix.1.queue0.lro_flushed: 0
dev.ix.1.queue1.interrupt_rate: 500000
dev.ix.1.queue1.irqs: 971514
dev.ix.1.queue1.txd_head: 0
dev.ix.1.queue1.txd_tail: 0
dev.ix.1.queue1.tso_tx: 0
dev.ix.1.queue1.no_tx_dma_setup: 0
dev.ix.1.queue1.no_desc_avail: 0
dev.ix.1.queue1.tx_packets: 0
dev.ix.1.queue1.rxd_head: 768
dev.ix.1.queue1.rxd_tail: 767
dev.ix.1.queue1.rx_packets: 971520
dev.ix.1.queue1.rx_bytes: 347213640
dev.ix.1.queue1.rx_copies: 663480
dev.ix.1.queue1.lro_queued: 0
dev.ix.1.queue1.lro_flushed: 0
dev.ix.1.queue2.interrupt_rate: 500000
dev.ix.1.queue2.irqs: 2178838
dev.ix.1.queue2.txd_head: 0
dev.ix.1.queue2.txd_tail: 0
dev.ix.1.queue2.tso_tx: 0
dev.ix.1.queue2.no_tx_dma_setup: 0
dev.ix.1.queue2.no_desc_avail: 0
dev.ix.1.queue2.tx_packets: 0
dev.ix.1.queue2.rxd_head: 1816
dev.ix.1.queue2.rxd_tail: 1815
dev.ix.1.queue2.rx_packets: 2178840
dev.ix.1.queue2.rx_bytes: 1161300000
dev.ix.1.queue2.rx_copies: 1239240
dev.ix.1.queue2.lro_queued: 0
dev.ix.1.queue2.lro_flushed: 0
dev.ix.1.queue3.interrupt_rate: 500000
dev.ix.1.queue3.irqs: 2644559
dev.ix.1.queue3.txd_head: 0
dev.ix.1.queue3.txd_tail: 0
dev.ix.1.queue3.tso_tx: 0
dev.ix.1.queue3.no_tx_dma_setup: 0
dev.ix.1.queue3.no_desc_avail: 0
dev.ix.1.queue3.tx_packets: 0
dev.ix.1.queue3.rxd_head: 592
dev.ix.1.queue3.rxd_tail: 591
dev.ix.1.queue3.rx_packets: 2644560
dev.ix.1.queue3.rx_bytes: 1756521840
dev.ix.1.queue3.rx_copies: 1032240
dev.ix.1.queue3.lro_queued: 0
dev.ix.1.queue3.lro_flushed: 0
dev.ix.1.queue4.interrupt_rate: 500000
dev.ix.1.queue4.irqs: 1001156
dev.ix.1.queue4.txd_head: 0
dev.ix.1.queue4.txd_tail: 0
dev.ix.1.queue4.tso_tx: 0
dev.ix.1.queue4.no_tx_dma_setup: 0
dev.ix.1.queue4.no_desc_avail: 0
dev.ix.1.queue4.tx_packets: 0
dev.ix.1.queue4.rxd_head: 1736
dev.ix.1.queue4.rxd_tail: 1735
dev.ix.1.queue4.rx_packets: 1001160
dev.ix.1.queue4.rx_bytes: 566511600
dev.ix.1.queue4.rx_copies: 527520
dev.ix.1.queue4.lro_queued: 0
dev.ix.1.queue4.lro_flushed: 0
dev.ix.1.queue5.interrupt_rate: 500000
dev.ix.1.queue5.irqs: 1141199
dev.ix.1.queue5.txd_head: 0
dev.ix.1.queue5.txd_tail: 0
dev.ix.1.queue5.tso_tx: 0
dev.ix.1.queue5.no_tx_dma_setup: 0
dev.ix.1.queue5.no_desc_avail: 0
dev.ix.1.queue5.tx_packets: 0
dev.ix.1.queue5.rxd_head: 464
dev.ix.1.queue5.rxd_tail: 463
dev.ix.1.queue5.rx_packets: 1141200
dev.ix.1.queue5.rx_bytes: 759726240
dev.ix.1.queue5.rx_copies: 564600
dev.ix.1.queue5.lro_queued: 0
dev.ix.1.queue5.lro_flushed: 0
dev.ix.1.queue6.interrupt_rate: 500000
dev.ix.1.queue6.irqs: 1361040
dev.ix.1.queue6.txd_head: 0
dev.ix.1.queue6.txd_tail: 0
dev.ix.1.queue6.tso_tx: 0
dev.ix.1.queue6.no_tx_dma_setup: 0
dev.ix.1.queue6.no_desc_avail: 0
dev.ix.1.queue6.tx_packets: 0
dev.ix.1.queue6.rxd_head: 1168
dev.ix.1.queue6.rxd_tail: 1167
dev.ix.1.queue6.rx_packets: 1361040
dev.ix.1.queue6.rx_bytes: 772066560
dev.ix.1.queue6.rx_copies: 482640
dev.ix.1.queue6.lro_queued: 0
dev.ix.1.queue6.lro_flushed: 0
dev.ix.1.queue7.interrupt_rate: 500000
dev.ix.1.queue7.irqs: 1200826
dev.ix.1.queue7.txd_head: 0
dev.ix.1.queue7.txd_tail: 0
dev.ix.1.queue7.tso_tx: 0
dev.ix.1.queue7.no_tx_dma_setup: 0
dev.ix.1.queue7.no_desc_avail: 0
dev.ix.1.queue7.tx_packets: 0
dev.ix.1.queue7.rxd_head: 712
dev.ix.1.queue7.rxd_tail: 711
dev.ix.1.queue7.rx_packets: 1200840
dev.ix.1.queue7.rx_bytes: 744130800
dev.ix.1.queue7.rx_copies: 660720
dev.ix.1.queue7.lro_queued: 0
dev.ix.1.queue7.lro_flushed: 0
dev.ix.1.mac_stats.crc_errs: 0
dev.ix.1.mac_stats.ill_errs: 0
dev.ix.1.mac_stats.byte_errs: 0
dev.ix.1.mac_stats.short_discards: 0
dev.ix.1.mac_stats.local_faults: 7
dev.ix.1.mac_stats.remote_faults: 2
dev.ix.1.mac_stats.rec_len_errs: 0
dev.ix.1.mac_stats.xon_txd: 0
dev.ix.1.mac_stats.xon_recvd: 0
dev.ix.1.mac_stats.xoff_txd: 0
dev.ix.1.mac_stats.xoff_recvd: 0
dev.ix.1.mac_stats.total_octets_rcvd: 6831554760
dev.ix.1.mac_stats.good_octets_rcvd: 6831554760
dev.ix.1.mac_stats.total_pkts_rcvd: 12000000
dev.ix.1.mac_stats.good_pkts_rcvd: 12000000
dev.ix.1.mac_stats.mcast_pkts_rcvd: 12480
dev.ix.1.mac_stats.bcast_pkts_rcvd: 33720
dev.ix.1.mac_stats.rx_frames_64: 305160
dev.ix.1.mac_stats.rx_frames_65_127: 4237560
dev.ix.1.mac_stats.rx_frames_128_255: 2372520
dev.ix.1.mac_stats.rx_frames_256_511: 252720
dev.ix.1.mac_stats.rx_frames_512_1023: 1553160
dev.ix.1.mac_stats.rx_frames_1024_1522: 3278880
dev.ix.1.mac_stats.recv_undersized: 0
dev.ix.1.mac_stats.recv_fragmented: 0
dev.ix.1.mac_stats.recv_oversized: 0
dev.ix.1.mac_stats.recv_jabberd: 0
dev.ix.1.mac_stats.management_pkts_rcvd: 0
dev.ix.1.mac_stats.management_pkts_drpd: 0
dev.ix.1.mac_stats.checksum_errs: 0
dev.ix.1.mac_stats.good_octets_txd: 0
dev.ix.1.mac_stats.total_pkts_txd: 0
dev.ix.1.mac_stats.good_pkts_txd: 0
dev.ix.1.mac_stats.bcast_pkts_txd: 0
dev.ix.1.mac_stats.mcast_pkts_txd: 0
dev.ix.1.mac_stats.management_pkts_txd: 0
dev.ix.1.mac_stats.tx_frames_64: 0
dev.ix.1.mac_stats.tx_frames_65_127: 0
dev.ix.1.mac_stats.tx_frames_128_255: 0
dev.ix.1.mac_stats.tx_frames_256_511: 0
dev.ix.1.mac_stats.tx_frames_512_1023: 0
dev.ix.1.mac_stats.tx_frames_1024_1522: 0
PS: I'm not using zerocopy yet (that's the next step and part of the
reason for my tests in the first place).
# sysctl -a | grep bpf
device bpf
net.bpf.maxinsns: 512
net.bpf.zerocopy_enable: 0
net.bpf.optimize_writers: 0
net.bpf.bufsize: 4194304
net.bpf.maxbufsize: 2138815488
I hope this information help you understand what is going on.
/Elof
On Wed, 4 Nov 2015, Christian Peron wrote:
> Your assumptions are correct, recv count is the number of packets which are received by the bpf peer, match count is the number of packets which matched the bpf filter that is active.. this shouldn’t be happening. Are you running netstat during the tcpreplay run or after it completed? Also can you give me more information on what specifically mon0 is?
>
>
>> On Nov 4, 2015, at 10:44 AM, elof2 at sentor.se wrote:
>>
>>
>> Hi!
>>
>> Question:
>> What do the Recv column in 'netstat -B' show?
>>
>> I thought it was tha amount of packets received, but appaently not so.
>>
>> I send 2000000 packets from a tcpreplay machine to a receiving machine.
>> I do it a few times.
>>
>> On the receiver I see:
>> netstat -in
>> Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll
>> ix0 1500 <Link#1> 0c:c4:7a:58:e2:3c 0 0 0 0 0 0
>> ix1 1500 <Link#2> 0c:c4:7a:58:e2:3d 6000000 0 0 0 0 0
>>
>> and then
>> netstat -in
>> Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll
>> ix0 1500 <Link#1> 0c:c4:7a:58:e2:3c 0 0 0 0 0 0
>> ix1 1500 <Link#2> 0c:c4:7a:58:e2:3d 8000000 0 0 0 0 0
>>
>> So 6000000 has increased to 8000000. Good.
>>
>>
>> However, 'netstat -B' show:
>> Pid Netif Flags Recv Drop Match Sblen Hblen Command
>> 25553 mon0 p--s--- 1996862 0 2000000 0 0 tcpdump
>>
>> How can the "Recv" be *lower* than "Match"?
>> 1996862 < 2000000.
>>
>>
>> For every new run (fast and slow) I get the same results, slightly less than 2000000 Recv.
>>
>> What am I missing?
>>
>> /Elof
>> _______________________________________________
>> freebsd-net at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
More information about the freebsd-net
mailing list