Strange behavior of forwarding between igb and ix on 11-STABLE: a lot of droppet packets no matter which pps is used

[Lev Serebryakov]

Hello Freebsd-net,

 I'm setting up simple experiment:

 Host A with em0  and ix0 (12.BETA2)
 Host B with igb1 and ix0 (11.2-STABLE)

 A:em0 is connected to B:igb1
 A:ix0 is connected to B:ix0

 Host B has static ARP for 192.168.10.2-192.168.10.9 set to A:ix0's MAC.
 Host B has forwarding enabled.

 I run packet generator on A:em0 to send packets for A:ix0 IP's
 (192.168.10.2-192.168.10.9) via host B with rate 64 pps (yes, pps):

A# pkt-gen -f tx -i em0 -d 192.168.10.2:2000-192.168.10.9 -D <B:igb1> -R 64

 I also run packet receiver on A:ix0 to measure these 64 pps.

A# pkt-gen -f rx -i ix0

 And receiver shows about 1/3 of pps! ~22pps!

 At host B "netstat -sp ip" shows:

        1790 packets forwarded (1790 packets fast forwarded)
        44 packets not forwardable
        0 packets received for unknown multicast group
        0 redirects sent
        1295 packets sent from this host
        0 packets sent with fabricated ip header
        1784 output packets dropped due to no bufs, etc.

 1790 forwarded packets are EXACTLY what receiver has been seen!

 And "1784 output packets dropped due to no bufs, etc." is counting when
 packets are forwarded, but not when host B sends traffic by itself

 If I increase PPS picture will be the same: only between 1/4 and 1/3 of all
 packets are forwarded.

 What do I do wrong?

 This ix0-ix0 link is fully capable of full 10G/s transfers when it is
 tested end-to-end, without any problems. If I test UDP transfer from B to A
 these "output packets dropped" are NOT counting even at 100Kpps! Interface is
 perfectly Ok!

-- 
Best regards,
 Lev                          mailto:lev at FreeBSD.org