ipfw nat and smaller wan mtu

From: John Hay <john_at_sanren.ac.za>
Date: Wed, 07 Dec 2022 14:33:47 UTC
Hi,

What would the proper ipfw rules be to make nat work and properly get the
icmp too big packets back to a local host if the wan interface needs a
smaller mtu?

I'm using a FreeBSD machine as router/firewall, but its wan interface needs
a smaller mtu (1392) than the default ethernet mtu. I have replicated this
in a VM so I can test it. My simplified ipfw rules make it work for packets
that are smaller than the wan mtu:

#####
net.inet.ip.fw.one_pass=0
net.inet.ip.fw.verbose=1
#####
fwcmd="/sbin/ipfw -q"
wan="vtnet0"
lan="vtnet1"
${fwcmd} nat 123 config if ${wan} log
${fwcmd} add 1000 count log all from any to any
${fwcmd} add 5000 nat 123 ip4 from any to any via ${wan}
${fwcmd} add 6000 allow log all from any to any
#####
The wan ip of the firewall is 10.10.2.2 and the ip address of the host (on
the lan side) I'm testing from is 10.10.1.3. And I did a ping to 10.10.5.5,
which is on the other side of the wan interface.

This works for packets smaller than the wan mtu. But if the packet is
larger than the wan mtu, the icmp too big is generated, but with 127.0.0.1
as the source and the wan ip as the destination and then sent via lo0 and
it looks like this in the ipfw log:

Dec  7 13:24:59 rtr kernel: ipfw: 1000 Count ICMP:3.4 127.0.0.1 10.10.2.2
out via lo0

So I added a nat ipfw rule to catch that:

${fwcmd} add 5050 nat 123 ip4 from any to not 127.0.0.1 via lo0

That helped partly because it was then able to recover the address of the
host I was testing from and tried to send the packet out on the correct
interface (vtnet1). Unfortunately it still had the source address of
127.0.0.1, which means it did not actually make it to the wire:

######
Dec  7 14:17:31 rtr kernel: ipfw: 1000 Count ICMP:8.0 10.10.1.3 10.10.5.5
in via vtnet1
Dec  7 14:17:31 rtr kernel: ipfw: 6000 Accept ICMP:8.0 10.10.1.3 10.10.5.5
in via vtnet1
Dec  7 14:17:31 rtr kernel: ipfw: 1000 Count ICMP:8.0 10.10.1.3 10.10.5.5
out via vtnet0
Dec  7 14:17:31 rtr kernel: ipfw: 6000 Accept ICMP:8.0 10.10.2.2 10.10.5.5
out via vtnet0
Dec  7 14:17:31 rtr kernel: ipfw: 1000 Count ICMP:3.4 127.0.0.1 10.10.2.2
out via lo0
Dec  7 14:17:31 rtr kernel: ipfw: 6000 Accept ICMP:3.4 127.0.0.1 10.10.2.2
out via lo0
Dec  7 14:17:31 rtr kernel: ipfw: 1000 Count ICMP:3.4 127.0.0.1 10.10.2.2
in via lo0
Dec  7 14:17:31 rtr kernel: ipfw: 6000 Accept ICMP:3.4 127.0.0.1 10.10.1.3
in via lo0
Dec  7 14:17:31 rtr kernel: ipfw: 1000 Count ICMP:3.4 127.0.0.1 10.10.1.3
out via vtnet1
Dec  7 14:17:31 rtr kernel: ipfw: 6000 Accept ICMP:3.4 127.0.0.1 10.10.1.3
out via vtnet1
######

Once I have this sorted, there seems to be a similar problem with nptv6.

Regards

John