[Bug 237853] pfsync does not sync state's "tag"?

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237853

            Bug ID: 237853
           Summary: pfsync does not sync state's "tag"?
           Product: Base System
           Version: 11.2-RELEASE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs at FreeBSD.org
          Reporter: johan at stromnet.se

In short, if a state is received via pfsync, and it's associated with a rule
that has a tag, should that tag be applied to matching packet(s) the same way
it would have been if the state was used on the original machine creating the
state?

It seems that this is not the case. pfsync_state_import
(https://github.com/freebsd/freebsd/blob/master/sys/netpfil/pf/if_pfsync.c#L459)
does not seem to set pf_state->tag, something which pf_create_state does on the
origin machine
(https://github.com/freebsd/freebsd/blob/master/sys/netpfil/pf/pf.c#L3786).

Not sure if this is a bug or just me missing something, but looking at the code
it seems it  *could* be something that's missing, unless there is an
undocumented (?) design decision.




Longer version:

I got two 11.2 machines (gw1, gw2) doing routing with vlans, carp interfaces,
PF and PFSync. The PF rulesets are in sync (same checksum). There are multiple
vlans, and one vlan (vlan2) is just used between gw1 and gw2 for pfsync, and in
some cases some traffic exchange due to potential asymmetric routing.

In the example below, we also have a machine "srv1", 172.28.1.2/24 on vlan10,
which uses the carp IP of gw1/2 as default gateway (here gw1 is master).

The problematic scenario is when gw2 (backup) tries to communicate with srv1,
and is sending from a "loopback" IP 172.28.3.249/30 (this also happens to be
the one on vlan2, I use that to universally reach the box, regardless from
which network).

gw2 will send UDP directly out on vlan10, but when srv1 replies to
172.28.3.249, it uses it's default gw so it ends up at gw1. Then gw1 is
supposed to forward this to gw2 via vlan2.

The outbound traffic from gw2 reaches srv1, and the return traffic from srv1
reaches gw1. But here it is *partially* blocked by pf.

I have the following rule (148) to allow this traffic out from (in this case)
gw2, and some other relevant rules:

@2 block return log on vlan2 all
@117 pass out all flags S/SA keep state tagged FWD
@148 pass out on vlan10 inet proto udp from any to 172.28.1.2 port = 1514 keep
state (udp.first 1000, udp.single 1000, udp.multiple 1000) tag FWD

The ideas is that the outbound packet on vlan10 creates a state which has the
tag FWD, and if needed the potential return traffic via the other gw, can be
passed out on vlan2.

A state gets created on gw2, and is synced fine to gw1 (with same rule ID in
pfctl -vvss)

As seen on gw1:
all udp 172.28.3.249:29940 -> 172.28.1.2:1514       MULTIPLE:MULTIPLE
   age 00:20:21, expires in 00:16:21, 0:3 pkts, 0:335 bytes, rule 148
   id: 010000005cb20e38 creatorid: 14f0eda0

As seen on gw2:
all udp 172.28.3.249:29940 -> 172.28.1.2:1514       MULTIPLE:MULTIPLE
   age 00:20:21, expires in 00:16:20, 41:2 pkts, 9813:202 bytes, rule 148
   id: 010000005cb20e38 creatorid: 14f0eda0

When 172.8.1.2 is replying to 172.28.3.249, the state packet counter on gw1 is
incremented, indicating that it is matched.
The intention is that it should go out on vlan2 on gw1, to finally reach gw2.

The routing itself works, but it is blocked by rule2:

13:00:16.094793 rule 2/0(match): block out on vlan2: (tos 0x0, ttl 63, id
14679, offset 0, flags [none], proto UDP (17), length 101)
    172.28.1.2.1514 > 172.28.3.249.29940: [udp sum ok] UDP, length 73


As the packet has matched a state associated with rule @148, I would expect
that it was tagged with the FWD tag, and thus allowed out on vlan2 as per rule
@117.


A simple workaround in this particular case is to just make sure the traffic
origins from IP on vlan10, so it can reply directly, but in some other cases
this might be more tricky.

-- 
You are receiving this mail because:
You are the assignee for the bug.