From nobody Fri Mar 03 21:12:41 2023 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PT0zy3D9Vz3wLg1 for ; Fri, 3 Mar 2023 21:12:46 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PT0zx4P73z49NP; Fri, 3 Mar 2023 21:12:45 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Authentication-Results: mx1.freebsd.org; none Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id 323LCfBl067696; Fri, 3 Mar 2023 13:12:41 -0800 (PST) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id 323LCfn8067695; Fri, 3 Mar 2023 13:12:41 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <202303032112.323LCfn8067695@gndrsh.dnsmgr.net> Subject: Re: BPF to filter/mod ARP In-Reply-To: <2A57C212-8FF0-406F-9AE5-58DCA856D716@lurchi.franken.de> To: Michael Tuexen Date: Fri, 3 Mar 2023 13:12:41 -0800 (PST) CC: "Rodney W. Grimes" , "Scheffenegger, Richard" , "freebsd-net@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 4PT0zx4P73z49NP X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N > > On 3. Mar 2023, at 14:52, Rodney W. Grimes wrote: > > > >>> On 2. Mar 2023, at 18:20, Rodney W. Grimes wrote: > >>> > >>>>> On 2. Mar 2023, at 02:24, Rodney W. Grimes wrote: > >>>>> > >>>>>> Hi group, > >>>>>> > >>>>>> Maybe someone can help me with this question - as I am usually only > >>>>>> looking at L4 and the top side of L3 ;) > >>>>>> > >>>>>> In order to validate a peculiar switches behavior, I want to adjust some > >>>>>> fields in gracious arps sent out by an interface, after a new IP is > >>>>>> assigned or changed. > >>>>> > >>>>> Gracious or Gratuitous? > >>>>> > >>>>>> > >>>>>> I believe BPF can effectively filter on arbitrary bit patterns and > >>>>>> modify packets on the fly. > >>>>> > >>>>> It can. > >>>>> > >>>>>> > >>>>>> However, as ARP doesn't seem to be accessible in the ipfw > >>>>>> infrastructure, I was wondering how to go about setting up an BPF to > >>>>>> tweak (temporarily) some of these ARPs to validate how the switch will > >>>>>> behave. > >>>>> > >>>>> ipfw is IP firewall, a layer 3 function. Arp is a layer 2 protocol, > >>>>> so very hard to do much with it in ipfw, but perhaps the layer2 > >>>>> keyword, and some use of mac-type can get it to match an arp > >>>>> packet. Arp is ethernet type 0x806. > >>>>> > >>>>> ipfw add 111 count log all from any to any layer2 mac-type arp > >>>>> That does seem to work > >>>>> ipfw -a list 111 > >>>>> 00111 4 0 count log ip from any to any layer2 mac-type 0x0806 > >>>>> > >>>>> Also normally ipfw does NOT pick packets up early enough to see > >>>>> them, to get the layer2 option to work you need: > >>>>> sysctl net.link.ether.ipfw=1 so that the filters at ether_demux > >>>>> get turned on. > >>>>> > >>>>> So perhaps use a divert rule and send them to a socket where > >>>>> a program can mangle them, and then return them to ipfw > >>>>> and hopefully the kernel does what you want after that... > >>>> I thought that you receive/send an IP packet on a divert socket, not > >>>> an ethernet frame. Am I wrong? > >>> > >>> That is unclear to me, technically it should just be a binary > >>> blob and the kernel and userland just have to agree as to > >>> what it is. Understand that ipfw originally only had IP layer > >>> functionality. The ability to muck with layer2 was added > >>> later, so I suspect the documentation about what is sent > >>> over the divert socket may be out of date. Simple enough > >>> to test though, just setup as I show above only change > >>> to: > >>> ipfw add 111 divert 4444 all from any to any layer2 mac-type arp > >>> and write a program to dump what you get on the divert socket. > >>> I suspect you get an ethernet frame. > >>> > >>> And finally divert(4) says: NAME: divert kernel packet diversion mechanism > >>> That says packet, so again, IMHO, it should be arbitrary to what layer. > >>> It also later says "Divert sockets are similar to raw IP sockets", > >>> I think similar is the key aspect here, they are not identical. > >> I can confirm that using > >> sudo sysctl net.link.ether.ipfw=1 > >> sudo ipfw add 111 count log all from any to any layer2 mac-type arp > >> ... wait some time and observe ARP traffic via tcpdump > >> sudo ipfw show > >> 00111 22 0 count log logamount 5 ip from any to any layer2 mac-type 0x0806 > >> 65535 7892 849004 allow ip from any to any > >> So the rule is hit. > >> > >> However, now doing > >> sudo ipfw delete 111 > >> sudo ipfw add 111 divert 1234 all from any to any layer2 mac-type arp > >> ... wait some time and observe ARP traffic via tcpdump > >> tuexen@head:~ % sudo ipfw show > >> 00111 0 0 divert 1234 ip from any to any layer2 mac-type 0x0806 > >> 65535 10048 1000948 allow ip from any to any > >> So this time, rule 111 is not hit. I also ran > > > > Nice work, to me I would classify this behavior as some form of bug, > > the action verb of a rule in ipfw should in no way change what is matched > > by the rule filter. > > > > I am assuming you either had IPDIVERT compiled into your kernel, or you > > you had loaded the module, as you dont clearly state this. I am also > > uncertain on what the results are if you use the divert keyword without > > ipdivert.ko loaded, is it an error when the rule gets created, or is it > > silently ignored? > Before compiling IPDIVERT into the kernel, I got an error message. So I > used the following kernel config for the testing: > > tuexen@head:~ % cat freebsd-src/sys/arm64/conf/TCP > include GENERIC > ident TCP > > makeoptions WITH_EXTRA_TCP_STACKS=1 > options TCPHPTS > options VIMAGE > options TCP_BLACKBOX > options TCPPCAP > options SCTP_DEBUG > options RATELIMIT > options DEBUG_REDZONE > options IPFIREWALL > options IPFIREWALL_VERBOSE > options IPFIREWALL_VERBOSE_LIMIT=5 > options IPFIREWALL_DEFAULT_TO_ACCEPT > options IPDIVERT And I did some further testing, if you try to add a "divert" rule without IPDIVERT either compiled into the kernel or loaded as a module you infact due get an error that the rule could not be added. I then went digging in the ether_demux code trying to find where ipfw (pfil in the kernel) gets ahold of the packet, did not find it in ether_demux, and the packet has been handled off to to the netisr code, and that is where I stopped in trying to find the path. I still find it very strange that a count rule shows packets, but no bytes, and a divert rule shows nothing. I suspect the divert rule is not getting a proper call to the pfil code to hook up the intercept. And a count rule probably only knows how to count IP payloads bytes. > > Best regards > Michael > > > >> > >> #include > >> #include > >> #include > >> #include > >> #include > >> #include > >> > >> #define BUFFER_SIZE (1<<16) > >> #define PORT 1234 > >> > >> int > >> main(void) > >> { > >> char buffer[BUFFER_SIZE]; > >> struct sockaddr_in addr; > >> ssize_t n; > >> int fd; > >> > >> if ((fd = socket(PF_DIVERT, SOCK_RAW, 0)) < 0) { > >> perror("socket()"); > >> } > >> bzero(&addr, sizeof(addr)); > >> addr.sin_family = AF_INET; > >> addr.sin_len = sizeof(struct sockaddr_in); > >> addr.sin_addr.s_addr = INADDR_ANY; > >> addr.sin_port = htons(PORT); > >> > >> if (bind(fd, (struct sockaddr *)&addr, (socklen_t)sizeof(struct sockaddr_in)) < 0) { > >> perror("bind()"); > >> } > >> for (;;) { > >> n = recv(fd, buffer, sizeof(buffer), 0); > >> printf("Received %zd bytes.\n", n); > >> } > >> if (close(fd) < 0) { > >> perror("close()"); > >> } > >> return (0); > >> } > >> > >> but nothing was printed... > >> > >> Best regards > >> Michael > >>> > >>>> > >>>> Best regards > >>>> Michael > >>>>> > >>>>>> (I need to validate, if there is some difference when the target > >>>>>> hardware address doesn't conform to RFC5227 - which states it SHOULD be > >>>>>> zero and is ignored on the receiving side; i have reasons to believe > >>>>>> that the switch needs either a target hardware address of > >>>>>> ff:ff:ff:ff:ff:ff or the local interface MAC, to properly update it's > >>>>>> entries.) > >>>>>> > >>>>>> Thanks a lot! > >>>>>> > >>>>>> Richard > >>>>>> > >>>>> > >>>>> -- > >>>>> Rod Grimes rgrimes@freebsd.org > >>>>> > >>>> > >>>> > >>>> > >>> > >>> -- > >>> Rod Grimes rgrimes@freebsd.org > >> > >> > >> > >> > > > > -- > > Rod Grimes rgrimes@freebsd.org > > > -- Rod Grimes rgrimes@freebsd.org