netmap nic to host stack forwarding

[Xiaoye Sun]

Hi,

I am new to netmap. What I want to do with netmap is to implement file
transfer protocol so that netmap can help achieve fast network IO. In the
netmap client program, I want that the unrelated packets are forwarded to
the linux host stack and all the packets from the host stack are forwarded
to the NIC also. That is, besides the netmap client achieving our file
transfer protocol, all the other netowrk applications running on top of the
host stack are not affected.

Our machine configuration is as follows:
Linux 3.11 + Myricom 10G nic + original Myricom driver coming in the linux
kernel, not the modified one.

We start to play with the bridge.c example in the netmap google code
repository. We want to use it in a way so that the netmap client forwards
packets from nic to the host stack and vice versa. So we use the command as
follows
sudo ./bridge -i netmap:eth3
to run the bridge netmap client. We did not change anything in the bridge.c
code and the netmap source code.

We test the functionality with the ping command line in linux. In our
setting, another machine (not running netmap) sends ping request message to
the netmap client machine and waits for the reply message. By looking at
the tcpdump, we found that all the packets got a copy. That is, in the
tcpdump results, there is a duplicate for each packet.

Here is the output of tcpdump -i eth3:

17:40:32.708585 IP 10.10.10.74 > 10.10.10.52: ICMP echo request, id 19910,
seq 1, length 64

17:40:32.708647 IP 10.10.10.74 > 10.10.10.52: ICMP echo request, id 19910,
seq 1, length 64

17:40:32.708721 IP 10.10.10.52 > 10.10.10.74: ICMP echo reply, id 19910,
seq 1, length 64

17:40:32.708760 IP 10.10.10.52 > 10.10.10.74: ICMP echo reply, id 19910,
seq 1, length 64

17:40:33.708410 IP 10.10.10.74 > 10.10.10.52: ICMP echo request, id 19910,
seq 2, length 64

17:40:33.708454 IP 10.10.10.74 > 10.10.10.52: ICMP echo request, id 19910,
seq 2, length 64

17:40:33.708507 IP 10.10.10.52 > 10.10.10.74: ICMP echo reply, id 19910,
seq 2, length 64

17:40:33.708540 IP 10.10.10.52 > 10.10.10.74: ICMP echo reply, id 19910,
seq 2, length 64

17:40:34.708400 IP 10.10.10.74 > 10.10.10.52: ICMP echo request, id 19910,
seq 3, length 64

17:40:34.708441 IP 10.10.10.74 > 10.10.10.52: ICMP echo request, id 19910,
seq 3, length 64

17:40:34.708488 IP 10.10.10.52 > 10.10.10.74: ICMP echo reply, id 19910,
seq 3, length 64

17:40:34.708516 IP 10.10.10.52 > 10.10.10.74: ICMP echo reply, id 19910,
seq 3, length 64


Here is the output of the bridge.c in verbose mode

420.122815 main [749] poll timeout [0] ev 1 0 rx 0 at 0 tx 1023, [1] ev 1 0 rx
0 at 0 tx 1023

422.625336 main [749] poll timeout [0] ev 1 0 rx 0 at 0 tx 1023, [1] ev 1 0 rx
0 at 0 tx 1023

425.127857 main [749] poll timeout [0] ev 1 0 rx 0 at 0 tx 1023, [1] ev 1 0 rx
0 at 0 tx 1023

427.630354 main [749] poll timeout [0] ev 1 0 rx 0 at 0 tx 1023, [1] ev 1 0 rx
0 at 0 tx 1023

430.132872 main [749] poll timeout [0] ev 1 0 rx 0 at 0 tx 1023, [1] ev 1 0 rx
0 at 0 tx 1023

432.635393 main [749] poll timeout [0] ev 1 0 rx 0 at 0 tx 1023, [1] ev 1 0 rx
0 at 0 tx 1023

432.708611 main [749] poll ok [0] ev 1 0 rx 0 at 0 tx 1023, [1] ev 1 1 rx 1 at 0
tx 1023

432.708635 main [749] poll ok [0] ev 5 4 rx 0 at 0 tx 1023, [1] ev 0 0 rx 1 at 0
tx 1023

432.708641 process_rings [562] net->host sent 1 packets to 0x7fef1aeaa000

432.708743 main [749] poll ok [0] ev 1 1 rx 1 at 0 tx 1023, [1] ev 1 0 rx 0 at 1
tx 1023

432.708751 main [749] poll ok [0] ev 0 0 rx 1 at 0 tx 1023, [1] ev 5 4 rx 0 at 1
tx 1023

432.708755 process_rings [562] net->host sent 1 packets to 0x7fef1aea1000

433.708430 main [749] poll ok [0] ev 1 0 rx 0 at 1 tx 1023, [1] ev 1 1 rx 1 at 1
tx 1022

433.708445 main [749] poll ok [0] ev 5 4 rx 0 at 1 tx 1023, [1] ev 0 0 rx 1 at 1
tx 1022

433.708449 process_rings [562] net->host sent 1 packets to 0x7fef1aeaa000

433.708525 main [749] poll ok [0] ev 1 1 rx 1 at 1 tx 1023, [1] ev 1 0 rx 0 at 2
tx 1022

433.708532 main [749] poll ok [0] ev 0 0 rx 1 at 1 tx 1023, [1] ev 5 4 rx 0 at 2
tx 1022

433.708536 process_rings [562] net->host sent 1 packets to 0x7fef1aea1000

434.708416 main [749] poll ok [0] ev 1 0 rx 0 at 2 tx 1023, [1] ev 1 1 rx 1 at 2
tx 1022

434.708432 main [749] poll ok [0] ev 5 4 rx 0 at 2 tx 1023, [1] ev 0 0 rx 1 at 2
tx 1022

434.708437 process_rings [562] net->host sent 1 packets to 0x7fef1aeaa000

434.708497 main [749] poll ok [0] ev 1 1 rx 1 at 2 tx 1023, [1] ev 1 0 rx 0 at 3
tx 1022

434.708505 main [749] poll ok [0] ev 0 0 rx 1 at 2 tx 1023, [1] ev 5 4 rx 0 at 3
tx 1022

434.708511 process_rings [562] net->host sent 1 packets to 0x7fef1aea1000

437.211025 main [749] poll timeout [0] ev 1 0 rx 0 at 3 tx 1023, [1] ev 1 0 rx
0 at 3 tx 1022

437.709106 main [749] poll ok [0] ev 1 1 rx 1 at 3 tx 1023, [1] ev 1 0 rx 0 at 3
tx 1022

437.709116 main [749] poll ok [0] ev 0 0 rx 1 at 3 tx 1023, [1] ev 5 4 rx 0 at 3
tx 1022

437.709121 process_rings [562] net->host sent 1 packets to 0x7fef1aea1000

437.709226 main [749] poll ok [0] ev 1 0 rx 0 at 4 tx 1023, [1] ev 1 1 rx 1 at 3
tx 1022

437.709235 main [749] poll ok [0] ev 5 4 rx 0 at 4 tx 1023, [1] ev 0 0 rx 1 at 3
tx 1022

437.709240 process_rings [562] net->host sent 1 packets to 0x7fef1aeaa000

440.211745 main [749] poll timeout [0] ev 1 0 rx 0 at 4 tx 1023, [1] ev 1 0 rx
0 at 4 tx 1022

442.714271 main [749] poll timeout [0] ev 1 0 rx 0 at 4 tx 1023, [1] ev 1 0 rx
0 at 4 tx 1022

444.026300 main [749] poll timeout [0] ev 1 0 rx 0 at 4 tx 1023, [1] ev 1 0 rx
0 at 4 tx 1022


I am wondering if the output looks normal. In particular, is this the
expected behavior of the bridge.c example?

In addition, how to make the netmap client only send one copy of the
packets to the network host stack and how to drop the packets in the netmap
client?

Do you think the problem is due to the unmodified Myricom driver?


Thanks!


Best,

Xiaoye

-- 
Xiaoye Steven Sun, Ph.D. Student
6100 Main St. MS-366
Department of Electrical and Computer Engineering (ECE) & Department of
Computer Science (CS)
George R. Brown School of Engineering
Rice University, Houston, Texas, USA