epair(4) + bridge(4) + pf(4) nat strangeness

[Nikolay Denev]

Hi,

I'm seeing something strange on my home router that I can't really
explain so any suggestions are welcome.

The machine is an Alix APU running

FreeBSD mars.home.lan 10.3-STABLE FreeBSD 10.3-STABLE #7: Wed May 18
19:03:58 UTC 2016     root at mars.home.lan:/usr/obj/usr/src/sys/MARS
amd64

It is compiled from svn rev 299223 with added the Codel+Pie patch for
ipfw which is not in use at the moment as ipfw has a single pass rule,
and the rest is in pf.conf (also in pf.conf I use ALTQ_CODEL)

re2 interface is connected to the ISP and pf.conf has "nat on re2"
statement, internal LAN network is connected to re0 and wireless
clients on wlan0, both are bridged in bridge0.
Since I wanted to run Suricata IDS for all internal traffic (both LAN
and WLAN), I have created a epair(4) interface, with one end added as
"span" port in bridge0, and the other I'm using in Suricata.

And here is where the strange stuff happens. For some reason on this
epair0b interface I'm seeing what it looks like duplicated traffic
from before and after being NATed.

For example, short tcpdump on epair0b shows this:


13:54:22.352206 IP (tos 0x0, ttl 63, id 29857, offset 0, flags [DF],
proto TCP (6), length 1480)
    10.0.0.13.51413 > XXX.XXX.XXX.XXX.12325: Flags [.], cksum 0xbca8
(correct), seq 59040:60480, ack 88, win 1035, length 1440

13:54:22.355368 IP (tos 0x0, ttl 63, id 29856, offset 0, flags [DF],
proto TCP (6), length 1480)
    ZZZ.ZZZ.ZZZ.ZZZ.51413 > XXX.XXX.XXX.XXX.12325: Flags [.], cksum
0x69d8 (correct), seq 59040:60480, ack 88, win 1035, length 1440

10.0.0.13 here is another FreeBSD box running transmission bt client,
and XXX.XXX.XXX.XXX is some random peer on the internet, but after
this I see on the interface the second packet which looks identical
with ip id minus one, and ZZZ.ZZZ.ZZZ.ZZZ is my public IP address
assigned to re2.

When doing tcpdump directly on bridge0, re0 or wlan0 I do not see this.


--Nikolay