netgraph arp issues vs linux veth

Tue Apr 27 01:15:02 PDT 2004

On Mon, Apr 26, 2004 at 11:22:43AM -0700, David Yeske wrote:
> I made another attempt with netgraph and I think I'm almost there, but I'm
> still having some issues.  I found a linux solution called veth
> http://www.geocities.com/nestorjpg/veth/ which might do the job, but I would
> prefer to use netgraph if possible.  Here is some more detailed config
> information.
> 
> I ran this on the spoof machine
> 
> # ngctl mkpeer . eiface hook ether
> # ifconfig ngeth0 link 00:bd:03:12:12:12
> # ifconfig ngeth0 192.168.10.3 netmask 255.255.255.0
> 
> # ngctl mkpeer ngeth0: bridge lower link0
> # ngctl name ngeth0:lower broken
> # ngctl connect fxp0: broken: lower link1
> # ngctl connect fxp0: broken: upper link2
> # ngctl connect ngeth0: broken: upper link3
> # ngctl msg ngeth0: setpromisc 1
> # ngctl msg ngeth0: setautosrc 0
> # ngctl msg fxp0: setpromisc 1
> # ngctl msg fxp0: setautosrc 0
> 
> # ngctl show broken:
>   Name: broken          Type: bridge          ID: 00000046   Num hooks: 4
>   Local hook      Peer name       Peer type    Peer ID         Peer hook
>   ----------      ---------       ---------    -------         ---------
>   link3           ngeth0          ether        00000005        upper
>   link2           fxp0            ether        00000004        upper
>   link1           fxp0            ether        00000004        lower
>   link0           ngeth0          ether        00000005        lower
> 
I experminted a bit more here, and it turns out that this can
be further simplified.  Instead of attaching both "lower" and
"upper' of ngeth0: ("ether" node type) to the bridge, you can
only attach "eiface" node's "ether" hook to the bridge (i.e.,
the corresponding "ether" node type will be left unconnected).
So bridge will be with three links: fxp0:lower, fxp0:upper,
and [eiface]:ether.

> on the remote machine an arp -a lists this
> ? (192.168.10.3) at 00:bd:03:12:12:12 on rl0 [ethernet] 
> ? (192.168.10.1) at 00:00:e8:5b:13:44 on rl0 permanent [ethernet]
> 
> on the spoof machine an arp -a lists this
> ? (192.168.10.1) at (incomplete) on ngeth0 [ethernet]
> ? (192.168.10.3) at 00:bd:03:12:12:12 on ngeth0 permanent [ethernet]
> 
> a sniff on the spoof machine listed this while pinging the remote machine
> 
> # tcpdump -i ngeth0 'ether host 00:00:e8:5b:13:44'
> tcpdump: listening on ngeth0
> 14:03:30.519263 arp reply 192.168.10.1 is-at 0:0:e8:5b:13:44
> 14:03:33.416568 192.168.10.1 > 192.168.10.3: icmp: echo request
> 14:03:40.530562 arp reply 192.168.10.1 is-at 0:0:e8:5b:13:44
> 14:03:43.427175 192.168.10.1 > 192.168.10.3: icmp: echo request
> 14:03:50.540805 arp reply 192.168.10.1 is-at 0:0:e8:5b:13:44
> 14:03:53.437845 192.168.10.1 > 192.168.10.3: icmp: echo request
> 14:04:00.550960 arp reply 192.168.10.1 is-at 0:0:e8:5b:13:44
> 14:04:03.448383 192.168.10.1 > 192.168.10.3: icmp: echo request
> 
> a sniff on the remote machine listed this while pinging the spoof machine
> 
> # tcpdump -i rl0 'ether host 00:bd:03:12:12:12'
> tcpdump: listening on rl0
> 14:02:24.918804 192.168.10.1 > 192.168.10.3: icmp: echo request
> 14:02:29.179263 arp reply 192.168.10.1 is-at 0:0:e8:5b:13:44
> 14:02:34.929051 192.168.10.1 > 192.168.10.3: icmp: echo request
> 14:02:44.939136 192.168.10.1 > 192.168.10.3: icmp: echo request
> 14:02:52.052260 arp reply 192.168.10.1 is-at 0:0:e8:5b:13:44
> 14:02:54.949402 192.168.10.1 > 192.168.10.3: icmp: echo request
> 14:03:02.063079 arp reply 192.168.10.1 is-at 0:0:e8:5b:13:44
> 14:03:04.959534 192.168.10.1 > 192.168.10.3: icmp: echo request
> 14:03:12.072830 arp reply 192.168.10.1 is-at 0:0:e8:5b:13:44
> 
> Any clues or pointers are greatly appreciated and will mean I get to deploy
> FreeBSD with netgraph rather than linux with veth.
> 
It seems that you use the same 192.168.10/24 network on both fxp0
and ngeth0 interfaces -- this is troublesome, as ARP in FreeBSD
(this is about to change soon) is the global, non per-interface
property, and is stored in a single routing table.  So when you
ping 192.168.10.3 from the remote box, it sends ARP request,
the bridge correctly forwards it to ngeth0, that correctly
replies with ARP reply, and ICMP ECHO REQUEST gets delivered to
ngeth0.  Then FreeBSD box with ngeth0 tries to send ICMP ECHO
REPLY.  For this, it looks up its routing table, and finds out
that 192.168.10.1 is on fxp0 (not on ngeth0!), so the reply
will be from the fxp0's MAC and with 192.168.10.3 source IP
address -- I've verified this locally.  Generally, it's not
possible to have two broadcast type interfaces in the same
subnetwork on one (FreeBSD) box.  Put it in other way: suppose
it works somehow.  So when you ping 192.168.10.1 from the
box with fxp0 and ngeth0, and both have addresses from the
same IP subnetwork, which interface (i.e., which MAC) should
get used?


Cheers,
-- 
Ruslan Ermilov
ru at FreeBSD.org
FreeBSD committer
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20040427/42239888/attachment.bin