9-STABLE + Infiniband - incorrect interface counters

[John Baldwin]

On Saturday, March 24, 2012 7:52:08 am Alex Tutubalin wrote:
> Hi,
> 
> I'm playing with two FreeBSD 9-STABLE boxes connected via 10Gbps 
> Infiniband (more details below) in Infiniband connected mode.
> 
> I see incorrect interface statistics (e.g. in netstat output), output 
> counters are 2x more than expected.
> 
> EXAMPLE, ftp transfer of 1 GiB file:
> 
> ftp> put file /dev/null
> local: file remote: /dev/null
> 229 Entering Extended Passive Mode (|||57978|)
> 150 Opening BINARY mode data connection for '/dev/null'.
> 100% |***********************************|   953 MiB  390.43 MiB/s    
> 00:00 ETA
> 226 Transfer complete.
> 1000000000 bytes sent in 00:02 (390.13 MiB/s)
> 
> Netstat on receiving side, counters are correct (for input):
> 
> lexa at home-gw:/home/lexa# netstat -I ib1 5
>              input          (ib1)           output
>     packets  errs idrops      bytes    packets  errs      bytes colls
>           0     0     0          0          0     0          0     0
>       13955     0     0  222688126       9027     0    1192796     0
>       48921     0     0  780832960      32129     0    4240596     0
>           0     0     0          0          0     0         80     0
> 
> Sum of bytes (input) is 1003521086, as expected.
> 
> Netstat on sending size, output is 2x more:
> 
> lexa at new-gw:/home/lexa# netstat -I ib0 5
>              input          (ib0)           output
>     packets  errs idrops      bytes    packets  errs      bytes colls
>           1     0     0        100          0     0          0     0
>       41162     0     0    2305210      62878     0 2008325984     0
>           1     0     0        100          0     0          0     0
> 
> It looks like packet count is correct (13955+48921=62876, two packets 
> missed somewhere), while byte count is exact 2x more.

Yes, this is a bug.  if_obytes already gets incremented in IFQ_HANDOFF(), so 
the IB code doesn't need to do it again.  Try the patch at 
www.freebsd.org/~jhb/patches/ipoib_obytes.patch

I can't speak to the MTU issue though.

-- 
John Baldwin