Netmap ixgbe stripping Vlan tags

Fri Aug 23 13:13:05 UTC 2013

Hi all,

I agree with Andre's statement 
> A netmap consumer
> typically doesn't expect packets be mangled at all, mostly likely netmap is
> expressly used to get the packet exactly as they were seen on the wire.

For my application I want to see the whole packet as is (as seen on the wire). 
I am sure it is important for many users who are interested in 
using netmap for speedup of packet capture in network security/monitoring applications.

When I disable "vlanhwfilter" flag on the interface. It is behaving as expected and is 
not stripping the Vlan tags when placed in promiscuous mode. Netmap seems to be ignoring 
his setting or is resetting this option  someplace (??). Any suggestion on where in Netmap 
code this maybe ?

Thanks for all your help.

Thanks,
Harika.

On Aug 23, 2013, at 7:36 AM, Andre Oppermann <andre at freebsd.org> wrote:

> On 23.08.2013 09:13, Juli Mallett wrote:
>> On Fri, Aug 23, 2013 at 12:02 AM, Andre Oppermann <andre at freebsd.org <mailto:andre at freebsd.org>> wrote:
>> 
>>    On 23.08.2013 00:36, Harika Tandra wrote:
>> 
>>        Hi all,
>> 
>>        I am running Netmap with "intel 10G 82598EB" card in promiscuous mode.
>>        While capturing packets via Netmap the driver is stripping off Vlan tags.
>>        I tested my setup, I am able to see Vlan tags when the same card is in promiscuous
>>        mode without Netmap.
>> 
>>        This maybe due to the netmap related changes to the device driver code.
>>        But I don't know much about drivers. I would appreciate any help or pointer
>>        regarding this.
>> 
>> 
>>    This is standard behavior in FreeBSD drivers where the vlan header is removed
>>    and the vlan tag is placed in an mbuf field.  Together with netmap it doesn't
>>    make sense of course and the driver should disable it when switching to netmap
>>    mode.
>> 
>> 
>> I think this runs counter to the netmap ethos some (as I understand it.)  In the same way that
>> netmap doesn't enable promiscuous mode since you may be doing non-promiscuous processing with
>> netmap, it also should leave whether VLAN_HWFILTER (or whatever) is enabled up to the program or the
>> user in question.  It would be nice if it could do the state management for these things (to ensure
>> the interface goes back to its original state if the program crashes), but as yet it can't.  There
>> are lots of passive capture applications where you might not care about having the VLAN tags intact
>> and so would prefer to have them stripped.  If there's a bug here, it's that packet metadata is lost
>> going into netmap entirely, not that the VLAN tags aren't in the frame.
> 
> I don't think vlan tag removal and promiscuous mode are comparable.  The former
> mangles the packet whereas the latter determines which packets are deliverd in
> the first place.  Running netmap w/o promiscuous mode makes a lot of sense when
> you only want to receive packets destined for this host.  A netmap consumer
> typically doesn't expect packets be mangled at all, mostly likely netmap is
> expressly used to get the packet exactly as they were seen on the wire.
> 
>>    The general usefulness of hardware vlan tag stripping/insertion is debatable as
>>    it doesn't gain much, if anything, and was intended for an entirely different
>>    purpose, namely to help the windows kernel over its total lack of vlan awareness.
>> 
>> 
>> It also helps (slightly) reduce overhead in packet processing where you just want to move data from
>> card to host as fast as possible.
> 
> There isn't much of a difference really.  Whether you do m_adj(m, -14) or m_adj(m, -18)
> is the same.  Parsing the tag out of the header is about the same as putting it in and
> reading it from the mbuf pkthdr.  The main overhead of passing it to the per-vlan ifnet
> is the same.  On the way out the same applies.  Prepending 14 vs. 18 bytes for the
> ethernet header isn't more overhead than what each driver has to do to for the vlan
> insertion descriptor setup.
> 
>>    It can be argued that in FreeBSD hardware vlan tag insert/removal is an unnecessary
>>    misfeature and only complicates things.  Especially in the context of multi-vlan
>>    stacking.  Doing it in software would be in fact just as fast remove a bit of code
>>    from the drivers.
>> 
>> Are you sure?  While that may hold up for ixgbe (though I'd say it doesn't, at least without packets
>> going straight into cache so there's no penalty for accessing parts of the packet you don't care
>> about, again looking at the netmap case really) it doesn't hold up for low-powered network
>> processors with very fast offload engines.
> 
> The stack has to look at the ethernet header anyway to determine what kind of
> packet it is.  The 4 additional bytes of vlan tag are in the same cache line.
> In the low-powered network processors with fast packet engines you try not to
> touch the packet on the CPU at all.  Even things like NAT are done in the engine.
> So you are limited to the capabilities of the hardware to retain full speed.
> 
> > (NB: I'd rather see it gone; I think it's an annoying
>> complication when writing network drivers [which have way too many fiddly bits; I was bound to
>> dislike some aspect of 'tools not policy' eventually, I guess],
> 
> Fully agreed here.  It only complicates things for no real benefit.  The way our
> stack works makes it totally unnecessary actually.
> 
> > or if it stays I'd rather see it be
>> a mandatory option on hardware where there's no penalty for using it.  Some hardware's performance
>> falls off spectacularly with this kind of offloading enabled, losing whatever gains stripping
>> unnecessary data provided.)
> 
> Having two path's makes things only more complicated.  While some NICs support
> QinQ stacking nowadays it makes integration into the stack yet more complex.
> I'd rather have it removed and do the magic in ether_input() at once and same
> for all.  Much less chance for bugs or surprising differences in behavior.
> Also allows for arbitrary deep QinQ stacking as has become all the rage in
> metro ethernet.
> 
> -- 
> Andre
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.freebsd.org/pipermail/freebsd-net/attachments/20130823/725c1600/attachment.sig>