Re: BPF to filter/mod ARP

From: Michael Tuexen <michael.tuexen_at_lurchi.franken.de>
Date: Fri, 03 Mar 2023 16:40:34 UTC
> On 3. Mar 2023, at 14:52, Rodney W. Grimes <freebsd-rwg@gndrsh.dnsmgr.net> wrote:
> 
>>> On 2. Mar 2023, at 18:20, Rodney W. Grimes <freebsd-rwg@gndrsh.dnsmgr.net> wrote:
>>> 
>>>>> On 2. Mar 2023, at 02:24, Rodney W. Grimes <freebsd-rwg@gndrsh.dnsmgr.net> wrote:
>>>>> 
>>>>>> Hi group,
>>>>>> 
>>>>>> Maybe someone can help me with this question - as I am usually only 
>>>>>> looking at L4 and the top side of L3 ;)
>>>>>> 
>>>>>> In order to validate a peculiar switches behavior, I want to adjust some 
>>>>>> fields in gracious arps sent out by an interface, after a new IP is 
>>>>>> assigned or changed.
>>>>> 
>>>>> Gracious or Gratuitous?
>>>>> 
>>>>>> 
>>>>>> I believe BPF can effectively filter on arbitrary bit patterns and 
>>>>>> modify packets on the fly.
>>>>> 
>>>>> It can.
>>>>> 
>>>>>> 
>>>>>> However, as ARP doesn't seem to be accessible in the ipfw 
>>>>>> infrastructure, I was wondering how to go about setting up an BPF to 
>>>>>> tweak (temporarily) some of these ARPs to validate how the switch will 
>>>>>> behave.
>>>>> 
>>>>> ipfw is IP firewall, a layer 3 function.  Arp is a layer 2 protocol,
>>>>> so very hard to do much with it in ipfw, but perhaps the layer2
>>>>> keyword, and some use of mac-type can get it to match an arp
>>>>> packet.  Arp is ethernet type 0x806.
>>>>> 
>>>>> ipfw add 111 count log all from any to any layer2 mac-type arp
>>>>> That does seem to work
>>>>> ipfw -a list 111
>>>>> 00111    4       0 count log ip from any to any layer2 mac-type 0x0806
>>>>> 
>>>>> Also normally ipfw does NOT pick packets up early enough to see
>>>>> them, to get the layer2 option to work you need:
>>>>> sysctl net.link.ether.ipfw=1 so that the filters at ether_demux
>>>>> get turned on.
>>>>> 
>>>>> So perhaps use a divert rule and send them to a socket where
>>>>> a program can mangle them, and then return them to ipfw
>>>>> and hopefully the kernel does what you want after that...
>>>> I thought that you receive/send an IP packet on a divert socket, not
>>>> an ethernet frame. Am I wrong?
>>> 
>>> That is unclear to me, technically it should just be a binary
>>> blob and the kernel and userland just have to agree as to
>>> what it is.  Understand that ipfw originally only had IP layer
>>> functionality.  The ability to muck with layer2 was added
>>> later, so I suspect the documentation about what is sent
>>> over the divert socket may be out of date.  Simple enough
>>> to test though, just setup as I show above only change
>>> to:
>>> ipfw add 111 divert 4444 all from any to any layer2 mac-type arp
>>> and write a program to dump what you get on the divert socket.
>>> I suspect you get an ethernet frame.
>>> 
>>> And finally divert(4) says: NAME: divert kernel packet diversion mechanism
>>> That says packet, so again, IMHO, it should be arbitrary to what layer.
>>> It also later says "Divert sockets are similar to raw IP sockets",
>>> I think similar is the key aspect here, they are not identical.
>> I can confirm that using
>> sudo sysctl net.link.ether.ipfw=1
>> sudo ipfw add 111 count log all from any to any layer2 mac-type arp
>> ... wait some time and observe ARP traffic via tcpdump
>> sudo ipfw show
>> 00111   22      0 count log logamount 5 ip from any to any layer2 mac-type 0x0806
>> 65535 7892 849004 allow ip from any to any
>> So the rule is hit.
>> 
>> However, now doing
>> sudo ipfw delete 111
>> sudo ipfw add 111 divert 1234 all from any to any layer2 mac-type arp
>> ... wait some time and observe ARP traffic via tcpdump
>> tuexen@head:~ % sudo ipfw show
>> 00111     0       0 divert 1234 ip from any to any layer2 mac-type 0x0806
>> 65535 10048 1000948 allow ip from any to any
>> So this time, rule 111 is not hit. I also ran
> 
> Nice work, to  me I would classify this behavior as some form of bug,
> the action verb of a rule in ipfw should in no way change what is matched
> by the rule filter.
> 
> I am assuming you either had IPDIVERT compiled into your kernel, or you
> you had loaded the module, as you dont clearly state this.   I am also
> uncertain on what the results are if you use the divert keyword without
> ipdivert.ko loaded, is it an error when the rule gets created, or is it
> silently ignored?
Before compiling IPDIVERT into the kernel, I got an error message. So I
used the following kernel config for the testing:

tuexen@head:~ % cat freebsd-src/sys/arm64/conf/TCP
include		GENERIC
ident	 	TCP

makeoptions     WITH_EXTRA_TCP_STACKS=1
options		TCPHPTS
options		VIMAGE
options		TCP_BLACKBOX
options		TCPPCAP
options		SCTP_DEBUG
options		RATELIMIT
options		DEBUG_REDZONE
options		IPFIREWALL
options		IPFIREWALL_VERBOSE
options		IPFIREWALL_VERBOSE_LIMIT=5
options		IPFIREWALL_DEFAULT_TO_ACCEPT
options		IPDIVERT

Best regards
Michael
> 
>> 
>> #include <sys/types.h>
>> #include <sys/socket.h>
>> #include <netinet/in.h>
>> #include <unistd.h>
>> #include <stdio.h>
>> #include <string.h>
>> 
>> #define BUFFER_SIZE (1<<16)
>> #define PORT        1234
>> 
>> int
>> main(void)
>> {
>> char buffer[BUFFER_SIZE];
>> struct sockaddr_in addr;
>> ssize_t n;
>> int fd;
>> 
>> if ((fd = socket(PF_DIVERT, SOCK_RAW, 0)) < 0) {
>> perror("socket()");
>> }
>> bzero(&addr, sizeof(addr));
>> addr.sin_family      = AF_INET;
>> addr.sin_len         = sizeof(struct sockaddr_in);
>> addr.sin_addr.s_addr = INADDR_ANY;
>> addr.sin_port        = htons(PORT);
>> 
>> if (bind(fd, (struct sockaddr *)&addr, (socklen_t)sizeof(struct sockaddr_in)) < 0) {
>> perror("bind()");
>> }
>> for (;;) {
>> n = recv(fd, buffer, sizeof(buffer), 0);
>> printf("Received %zd bytes.\n", n);
>> }
>> if (close(fd) < 0) {
>> perror("close()");
>> }
>> return (0);
>> }
>> 
>> but nothing was printed...
>> 
>> Best regards
>> Michael
>>> 
>>>> 
>>>> Best regards
>>>> Michael
>>>>> 
>>>>>> (I need to validate, if there is some difference when the target 
>>>>>> hardware address doesn't conform to RFC5227 - which states it SHOULD be 
>>>>>> zero and is ignored on the receiving side; i have reasons to believe 
>>>>>> that the switch needs either a target hardware address of 
>>>>>> ff:ff:ff:ff:ff:ff or the local interface MAC, to properly update it's 
>>>>>> entries.)
>>>>>> 
>>>>>> Thanks a lot!
>>>>>> 
>>>>>> Richard
>>>>>> 
>>>>> 
>>>>> -- 
>>>>> Rod Grimes                                                 rgrimes@freebsd.org
>>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>>> -- 
>>> Rod Grimes                                                 rgrimes@freebsd.org
>> 
>> 
>> 
>> 
> 
> -- 
> Rod Grimes                                                 rgrimes@freebsd.org