netmap over virtio giving packets with extra 12 bytes

[Luigi Rizzo]

On Tue, Jan 06, 2015 at 10:15:02AM -0800, Adrian Chadd wrote:
...
> This won't be the first time that there'll be useful data at the front
> end of an RX mbuf that isn't related to the mbuf payload.
> 
> It'd be nice if there were something in each rx ring slot saying how
> far to skip into the buffer to get the beginning of the packet.

I am not opposed in principle, this is something we have been looking
at since day one. The blocking issue is that incompatible hw constraints
make it hard to make a decent choice.

Examples:
1. the rx buffer size you tell to ixgbe must be a power of two.
  If you want to write at some offset into the netmap buffer,
  you need to allocate one twice the size you pass to the driver.
2. some NICs may want buffers aligned to 4, 8, 16 bytes, so input
  offsets for headers cannot be arbitrary (12 is almost as bad as 14!)
3. irrespective of functionality, performance drops badly with small
  packets (where it matters the most) when buffers are not aligned
  to 64 byte boundaries.

The above makes me think that for small packets, copying is the
only reasonable way to go, and for large packets i have no idea
how to deal with #1 and #2 without having to do scatter-gather.

If you have a good suggestion please speak up.

cheers
luigi