netmap design question - accessing netmap:X-n individual queues on FreeBSD

[Eduardo Meyer]

Hello all,

I have some doubts regarding netmap design direct queue usage.

If open netmap:ix0 I am opening all 0-7 queues. Are those queues FIFO among
themselves? I mean first packeds will be available on netmap:ix0-0 and if
this queue fills up the next packets will be on netmap:ix0-1, and via
netmap:ix0 I have all queues from 0 to 7, is this understanding correct?

Or something else happens, like, only 1 queue is used if I open netmap:ix0?

A later question, if I open netmap:ix0-0 and nothing else is it supposed to
work?

On my tests I can see that "pkt-gen -f tx -i ix0-0" will work, but "pkt-gen
-f rx -i ix0-0" will not. I can transmit but cant receive on a given queue.
Why is that? And how could I make something like this, work (code change
required?):

bridge -i netmap:ix0-0 -i netmap:ix1-0

Or should it already work?

Mr Pavel Odiltsov, the author from fastnetmon mentioned he can run on Linux
and it works:

"kipfw netmap:eth0-n netmap:eth1-n"

But this or the above bridge example won't work on FreeBSD. Is that any
different? (I did not try on Linux).

I could also notice performance differences I would like to understand, if
I run:

pkt-gen -i ix0 -f tx -s 192.168.0.2 -d 192.168.0.1

I have 14.8Mpps (like rate).

If I run:

pkt-gen -i ix0-1 -f tx -s 192.168.0.2 -d 192.168.0.1

I can have only 11Mpps. In fact I have 11Mpps if I run on ix0-2, ix0-3, ...
ix0-7.

I can understand if I run all queues I can have better pps rates than only
one single queue, sure, however if I run:

pkt-gen -i ix0-0 -f tx -s 192.168.0.2 -d 192.168.0.1

I also have 14.8Mpps. So yeah, ix0 or ix0-0 both give me 14.8Mpps while any
other queue give me 11Mpps. How should I understand this?

I am asking this because I want to hack (for learning) into the bridge code
to make it multithreaded, and I want to have a thread opening each one of
the 8 available queues allocated on each one of my 8 CPUs.

However both opening ix0-1 and ix1-1 on source code or as a parameter to
the bridge application, I can't have it working.

I also looked on "pkt-gen -p 8 -c 8" and although debug shows I have 8
threads on 8 CPU with 8 queues I still only see 1 thread working, all other
threads go IDLE. I expected to see at least 2 threads working, say, for
ix0-0 and ix0-1 to fill line rate.

Thank you in advance.

-- 
===========
Eduardo Meyer