cxgbetool & hw filtering issues

[Navdeep Parhar]

On 06/30/13 07:25, Alexander V. Chernikov wrote:
> Hello list!
> 
> While experimenting with Chelsio T440-CR (cxgbe) internal firewall, I'm
> getting some kind of unexpected results:

One bit of general advice to begin with: add "hitcnts 1" to all your
filter rules and then you can see how many incoming packets hit that
filter in the output of "cxgbetool t4nex0 filter list".  I really should
make hitcnts=1 the default in the driver.

> filtering 'type ipv4 action drop' permits IPv4 TCP traffic with bad
> checksum.

It may be that a bad checksum makes it an invalid IPv4 packet to the
chip and so it doesn't hit the "type ipv4" rule.  There is an entirely
separate knob available to have the chip drop bad packets if you don't
want to see them.  The default is to let them through so that users can
examine them with tcpdump etc.

> filtering 'type IPv6 action drop' permits IPv6 traffic to multicast
> addresses (MLDv2, etc..)

The DMAC is an L2 multicast address?  Try "proto 58 hitcnts 1 action
drop" to get these ICMP6 packets.

> filtering 'ethtype 34525 action drop' (drop all IPv6) results in
> 'CHELSIO_T4_SET_FILTER: Argument list too long' despite to what is said
> in budget table from cxgbetool.8

This _would_ have gotten everything with ethertype ipv6 but the default
filter mode doesn't have ethtype enabled, which is why it's complaining:
# cxgbetool t4nex0 filter mode
ipv4 ipv6 sip dip sport dport matchtype proto ivlan iport fcoe

> filtering 'matchtype 4 action drop' or similar (4,5,4:0,4:4, 5:0, 5:5)
> does not match anything despite some traffic definitely falls into that
> conditions.
> filtering 'action drop' and 'iport X action drop' filters IPv4 traffic
> only.

Strange.  I use "iport X action drop hitcnts 1" as a packet black hole
all the time.  Were these the only filters when you tried them?  Are you
sure your packets didn't hit some other rule and were delivered as a
result of that?  Check the order in "cxgbetool t4nex0 filter list"

Also, are you going by the ifnet rx stats as displayed by netstat etc.?
 Right now the driver fills the ifnet stats directly from hardware
registers rather than counting the packets that it actually received
from the chip.  The hardware registers include packets that would have
been delivered to the driver if no filters were present but are dropped
due to a filter.

> filter 'type ipv6 ...' can be set on (0,4,8,12,...) filter numbers
> yelling 'CHELSIO_T4_SET_FILTER: Invalid argument' on other numbers.

Yes, IPv6 filters take 4 tid's (non-IPv6 take 1) and these tid's have to
start at a naturally aligned boundary.  No way around this.

> What can I do to debug further/fix this behavior?
> 
> Some more questions:
> Does anybody known how I can get/set total number of HW firewall
> records? There is such tunable in Linux version.

I will add a simple sysctl for this.  For now you can indirectly figure
this out from the output of "sysctl -n dev.t4nex.0.misc.tids" -- the
FTIDs are the filter tids.  For example I see 1456 filters on this card:
trantor:~# sysctl -n dev.t4nex.0.misc.tids
ATID range: 0-8191, in use: 0
TID range: 2048-18431, in use: 0
STID range: 0-511, in use: 0
FTID range: 512-1967
HW TID usage: 0 IP users, 0 IPv6 users
trantor:~# echo $((1967 - 512 + 1))
1456

> Is there any way to retrieve _host_ interface statistic (e.g. how much
> traffic in packets/bytes are thrown to NIC driver)?

cxgbe(4) doesn't count this stuff itself.  Currently it just reads the
hardware registers once per second and it's done.  Software stats would
have to be per queue (and then aggregated from time to time).  I'll wait
to see where the PCPU counter work in the kernel goes before reworking
this part of the driver.

Regards,
Navdeep