PF ICMP generation

From: Daniel O'Connor <darius_at_dons.net.au>
Date: Sun, 18 Sep 2022 05:12:16 UTC
Hi,
I discovered recently my firewall rules were allowing NTP in and I had a misconfigured host which was spamming traffic. I ended up moving the host to a different IP (and fixing the firewall rule) but now I find the router (which I recently updated to FreeBSD 13.1-RELEASE-p2, from 12) is generating mounds of ICMP unreachable traffic:
14:10:30.572851 IP a.a.a.a > 201.150.126.195: ICMP b.b.b.b udp port 123 unreachable, length 44
14:10:30.572865 IP a.a.a.a > 201.150.127.67: ICMP b.b.b.b udp port 123 unreachable, length 44
14:10:30.572872 IP a.a.a.a > 201.150.126.195: ICMP b.b.b.b udp port 123 unreachable, length 44
14:10:30.573661 IP a.a.a.a > 201.150.126.67: ICMP b.b.b.b udp port 123 unreachable, length 44
14:10:30.574262 IP a.a.a.a > 201.150.126.195: ICMP b.b.b.b udp port 123 unreachable, length 44
14:10:30.574273 IP a.a.a.a > 201.150.127.67: ICMP b.b.b.b udp port 123 unreachable, length 44
14:10:30.574280 IP a.a.a.a > 201.150.127.67: ICMP b.b.b.b udp port 123 unreachable, length 44
14:10:30.575212 IP a.a.a.a > 201.150.127.67: ICMP b.b.b.b udp port 123 unreachable, length 44
14:10:30.575222 IP a.a.a.a > 201.150.127.195: ICMP b.b.b.b udp port 123 unreachable, length 44
14:10:30.575239 IP a.a.a.a > 201.150.126.195: ICMP b.b.b.b udp port 123 unreachable, length 44

Where a.a.a.a is the IP of the router (static from the ISP) and b.b.b.b is the (publicly routable) IP of the machine I changed the IP for.

The normal ICMP rate limiting does not seem to be working: net.inet.icmp.icmplim is the default of 200, and I do not see any "Limiting icmp unreach response from NNNN to 200 packets/sec" messages which I have definitely seen before.
Blocking the IPs it is sending to in pf does not seem to stop the traffic.

I then ran dtrace to try and work out which code was generating the ICMP traffic:
[portero 14:25] ~ >sudo dtrace -n 'fbt::icmp_error:entry { @stacks[stack()] = count(); }'
dtrace: description 'fbt::icmp_error:entry ' matched 1 probe
^C
              pf.ko`pf_intr+0x1e5
              kernel`fork_exit+0x7e
              kernel`0xffffffff810885ee
            15273

I then realised that my PF default block policy was 'return' rather than 'drop' - once I changed that it was good.

However I think it is pretty surprising that PF does not implement the ICMP rate limiter (net.inet.icmp.icmplim).

I checked the code and I think it would not be too difficult to add a called to badport_bandlim() to either pf_intr or pf_send_icmp but I haven't tried it yet.

--
Daniel O'Connor
"The nice thing about standards is that there
are so many of them to choose from."
-- Andrew Tanenbaum