Re: BPF to filter/mod ARP

From: Rodney W. Grimes <freebsd-rwg_at_gndrsh.dnsmgr.net>
Date: Fri, 03 Mar 2023 21:12:41 UTC
> > On 3. Mar 2023, at 14:52, Rodney W. Grimes <freebsd-rwg@gndrsh.dnsmgr.net> wrote:
> > 
> >>> On 2. Mar 2023, at 18:20, Rodney W. Grimes <freebsd-rwg@gndrsh.dnsmgr.net> wrote:
> >>> 
> >>>>> On 2. Mar 2023, at 02:24, Rodney W. Grimes <freebsd-rwg@gndrsh.dnsmgr.net> wrote:
> >>>>> 
> >>>>>> Hi group,
> >>>>>> 
> >>>>>> Maybe someone can help me with this question - as I am usually only 
> >>>>>> looking at L4 and the top side of L3 ;)
> >>>>>> 
> >>>>>> In order to validate a peculiar switches behavior, I want to adjust some 
> >>>>>> fields in gracious arps sent out by an interface, after a new IP is 
> >>>>>> assigned or changed.
> >>>>> 
> >>>>> Gracious or Gratuitous?
> >>>>> 
> >>>>>> 
> >>>>>> I believe BPF can effectively filter on arbitrary bit patterns and 
> >>>>>> modify packets on the fly.
> >>>>> 
> >>>>> It can.
> >>>>> 
> >>>>>> 
> >>>>>> However, as ARP doesn't seem to be accessible in the ipfw 
> >>>>>> infrastructure, I was wondering how to go about setting up an BPF to 
> >>>>>> tweak (temporarily) some of these ARPs to validate how the switch will 
> >>>>>> behave.
> >>>>> 
> >>>>> ipfw is IP firewall, a layer 3 function.  Arp is a layer 2 protocol,
> >>>>> so very hard to do much with it in ipfw, but perhaps the layer2
> >>>>> keyword, and some use of mac-type can get it to match an arp
> >>>>> packet.  Arp is ethernet type 0x806.
> >>>>> 
> >>>>> ipfw add 111 count log all from any to any layer2 mac-type arp
> >>>>> That does seem to work
> >>>>> ipfw -a list 111
> >>>>> 00111    4       0 count log ip from any to any layer2 mac-type 0x0806
> >>>>> 
> >>>>> Also normally ipfw does NOT pick packets up early enough to see
> >>>>> them, to get the layer2 option to work you need:
> >>>>> sysctl net.link.ether.ipfw=1 so that the filters at ether_demux
> >>>>> get turned on.
> >>>>> 
> >>>>> So perhaps use a divert rule and send them to a socket where
> >>>>> a program can mangle them, and then return them to ipfw
> >>>>> and hopefully the kernel does what you want after that...
> >>>> I thought that you receive/send an IP packet on a divert socket, not
> >>>> an ethernet frame. Am I wrong?
> >>> 
> >>> That is unclear to me, technically it should just be a binary
> >>> blob and the kernel and userland just have to agree as to
> >>> what it is.  Understand that ipfw originally only had IP layer
> >>> functionality.  The ability to muck with layer2 was added
> >>> later, so I suspect the documentation about what is sent
> >>> over the divert socket may be out of date.  Simple enough
> >>> to test though, just setup as I show above only change
> >>> to:
> >>> ipfw add 111 divert 4444 all from any to any layer2 mac-type arp
> >>> and write a program to dump what you get on the divert socket.
> >>> I suspect you get an ethernet frame.
> >>> 
> >>> And finally divert(4) says: NAME: divert kernel packet diversion mechanism
> >>> That says packet, so again, IMHO, it should be arbitrary to what layer.
> >>> It also later says "Divert sockets are similar to raw IP sockets",
> >>> I think similar is the key aspect here, they are not identical.
> >> I can confirm that using
> >> sudo sysctl net.link.ether.ipfw=1
> >> sudo ipfw add 111 count log all from any to any layer2 mac-type arp
> >> ... wait some time and observe ARP traffic via tcpdump
> >> sudo ipfw show
> >> 00111   22      0 count log logamount 5 ip from any to any layer2 mac-type 0x0806
> >> 65535 7892 849004 allow ip from any to any
> >> So the rule is hit.
> >> 
> >> However, now doing
> >> sudo ipfw delete 111
> >> sudo ipfw add 111 divert 1234 all from any to any layer2 mac-type arp
> >> ... wait some time and observe ARP traffic via tcpdump
> >> tuexen@head:~ % sudo ipfw show
> >> 00111     0       0 divert 1234 ip from any to any layer2 mac-type 0x0806
> >> 65535 10048 1000948 allow ip from any to any
> >> So this time, rule 111 is not hit. I also ran
> > 
> > Nice work, to  me I would classify this behavior as some form of bug,
> > the action verb of a rule in ipfw should in no way change what is matched
> > by the rule filter.
> > 
> > I am assuming you either had IPDIVERT compiled into your kernel, or you
> > you had loaded the module, as you dont clearly state this.   I am also
> > uncertain on what the results are if you use the divert keyword without
> > ipdivert.ko loaded, is it an error when the rule gets created, or is it
> > silently ignored?
> Before compiling IPDIVERT into the kernel, I got an error message. So I
> used the following kernel config for the testing:
> 
> tuexen@head:~ % cat freebsd-src/sys/arm64/conf/TCP
> include		GENERIC
> ident	 	TCP
> 
> makeoptions     WITH_EXTRA_TCP_STACKS=1
> options		TCPHPTS
> options		VIMAGE
> options		TCP_BLACKBOX
> options		TCPPCAP
> options		SCTP_DEBUG
> options		RATELIMIT
> options		DEBUG_REDZONE
> options		IPFIREWALL
> options		IPFIREWALL_VERBOSE
> options		IPFIREWALL_VERBOSE_LIMIT=5
> options		IPFIREWALL_DEFAULT_TO_ACCEPT
> options		IPDIVERT

And I did some further testing, if you try to add a "divert"
rule without IPDIVERT either compiled into the kernel or
loaded as a module you infact due get an error that the
rule could not be added.  I then went digging in the
ether_demux code trying to find where ipfw (pfil in the
kernel) gets ahold of the packet, did not find it in
ether_demux, and the packet has been handled off to
to the netisr code, and that is where I stopped in
trying to find the path.

I still find it very strange that a count rule shows packets,
but no bytes, and a divert rule shows nothing.  I suspect
the divert rule is not getting a proper call to the pfil
code to hook up the intercept.  And a count rule probably
only knows how to count IP payloads bytes.

> 
> Best regards
> Michael
> > 
> >> 
> >> #include <sys/types.h>
> >> #include <sys/socket.h>
> >> #include <netinet/in.h>
> >> #include <unistd.h>
> >> #include <stdio.h>
> >> #include <string.h>
> >> 
> >> #define BUFFER_SIZE (1<<16)
> >> #define PORT        1234
> >> 
> >> int
> >> main(void)
> >> {
> >> char buffer[BUFFER_SIZE];
> >> struct sockaddr_in addr;
> >> ssize_t n;
> >> int fd;
> >> 
> >> if ((fd = socket(PF_DIVERT, SOCK_RAW, 0)) < 0) {
> >> perror("socket()");
> >> }
> >> bzero(&addr, sizeof(addr));
> >> addr.sin_family      = AF_INET;
> >> addr.sin_len         = sizeof(struct sockaddr_in);
> >> addr.sin_addr.s_addr = INADDR_ANY;
> >> addr.sin_port        = htons(PORT);
> >> 
> >> if (bind(fd, (struct sockaddr *)&addr, (socklen_t)sizeof(struct sockaddr_in)) < 0) {
> >> perror("bind()");
> >> }
> >> for (;;) {
> >> n = recv(fd, buffer, sizeof(buffer), 0);
> >> printf("Received %zd bytes.\n", n);
> >> }
> >> if (close(fd) < 0) {
> >> perror("close()");
> >> }
> >> return (0);
> >> }
> >> 
> >> but nothing was printed...
> >> 
> >> Best regards
> >> Michael
> >>> 
> >>>> 
> >>>> Best regards
> >>>> Michael
> >>>>> 
> >>>>>> (I need to validate, if there is some difference when the target 
> >>>>>> hardware address doesn't conform to RFC5227 - which states it SHOULD be 
> >>>>>> zero and is ignored on the receiving side; i have reasons to believe 
> >>>>>> that the switch needs either a target hardware address of 
> >>>>>> ff:ff:ff:ff:ff:ff or the local interface MAC, to properly update it's 
> >>>>>> entries.)
> >>>>>> 
> >>>>>> Thanks a lot!
> >>>>>> 
> >>>>>> Richard
> >>>>>> 
> >>>>> 
> >>>>> -- 
> >>>>> Rod Grimes                                                 rgrimes@freebsd.org
> >>>>> 
> >>>> 
> >>>> 
> >>>> 
> >>> 
> >>> -- 
> >>> Rod Grimes                                                 rgrimes@freebsd.org
> >> 
> >> 
> >> 
> >> 
> > 
> > -- 
> > Rod Grimes                                                 rgrimes@freebsd.org
> 
> 
> 

-- 
Rod Grimes                                                 rgrimes@freebsd.org