ixl rxq choice with QinQ frames

Hi folks,

I am trying to build a box to perform what can only be described as "L2 
tagging gymnastics" - essentially taking double-tagged frames in/out on 
one ixl interface, and doing some netgraph bridging to put them into a 
small subset (around 6) of single-tagged VLANs on another ixl 
interface.  This all lab-tested well with decent performance too.

However, when we tried to use it with real-world traffic, the throughput 
was poor and dev.ixl.1.pf.rx_discards was very high.  A closer look at 
the dev.ixl.1.pf.rxq*.packets output showed that pretty much all of the 
packets were rx on queue 9.  Some researching led me to realise that the 
QinQ frames couldn't be classified easily by the NIC, hence this outcome 
- and it explained the poor performance.

I've tried the usual: disable tso/lro + other hardware assistance on 
NIC, but still have the problem.

I have also tried a minor change to ixl_pf_main.c - to try and make it 
hash at L2, in ixl_set_rss_pctypes, I added 
I40E_FILTER_PCTYPE_L2_PAYLOAD to the set_hena value being computed.  
This made no difference, and after checking with the datasheet it became 
clear why: If I'm reading things correctly, the PCTYPE_L2_PAYLOAD causes 
it to hash on the L2 Ethertype (datasheet section 7.1.2); which for all 
of my frames will be 0x8100.

So I'm back to square one.  I guess I am looking for a way to either 
hash the queue choice on L2 src/dest MAC address, or even round-robin 
across the queues, to avoid all received frames being stuck on one queue.

Is there any other way to get decent performance when receiving QinQ 
frames on these NICs that I've missed?  Somewhat ironically, I chose ixl 
NICs for this use-case because of the decent number of tx and rx queues 
... but that doesn't help if they're not being used :-)

Thanks

Paul.