Major performance hit with ToS setting

[Kevin Oberman]

On Fri, May 25, 2012 at 6:27 AM, Andrew Gallatin <gallatin at cs.duke.edu> wrote:
> On 05/24/12 18:55, Kevin Oberman wrote:
>
>>
>> This is,of course, on a 10G interface. On 7.3 there is little
>
>
> Hi Kevin,
>
>
> What you're seeing looks almost like a checksum is bad, or
> there is some other packet damage.  Do you see any
> error counters increasing if you run netstat -s before
> and after the test & compare the results?
>
> Thinking that, perhaps, this was a bug in my mxge(4), I attempted
> to reproduce it this morning between  8.3 and 9.0 boxes and
> failed to see the bad behavior..
>
> % nuttcp-6.1.2 -c32t -t diablo1-m < /dev/zero
>  9161.7500 MB /  10.21 sec = 7526.5792 Mbps 53 %TX 97 %RX 0 host-retrans
> 0.11 msRTT
> % nuttcp-6.1.2  -t diablo1-m < /dev/zero
>  9140.6180 MB /  10.21 sec = 7509.8270 Mbps 53 %TX 97 %RX 0 host-retrans
> 0.11 msRTT
>
>
> However, I don't have any 8.2-r box handy, so I cannot
> exactly repro your experiment...

As seems to always be the case, the test system I set up to try other
versions of FreeBSD does not exhibit the problem. I have no difficulty
generating traffic at speeds above 1G. (I saw a max on one run over
just a few miles of over 9G.)

So I am scratching my head and re-testing this.

FWIW, three different engineers tested production systems and all saw
the same issue, whether using nuttcp or iperf. If I run hte tests now
on one of those systems, I see the problem. So I am off to do a bunch
more testing and I'll try to have some real information soon.

Thanks for looking into this.
-- 
R. Kevin Oberman, Network Engineer
E-mail: kob6558 at gmail.com