Terrible ix performance

[Outback Dingo]

On Wed, Jul 3, 2013 at 4:50 AM, Lawrence Stewart <lstewart at freebsd.org>wrote:

> On 07/03/13 14:28, Outback Dingo wrote:
> > Ive got a high end storage server here, iperf shows decent network io
> >
> > iperf -i 10 -t 20 -c 10.0.96.1 -w 2.5M -l 2.5M
> > ------------------------------------------------------------
> > Client connecting to 10.0.96.1, TCP port 5001
> > TCP window size: 2.50 MByte (WARNING: requested 2.50 MByte)
> > ------------------------------------------------------------
> > [  3] local 10.0.96.2 port 34753 connected with 10.0.96.1 port 5001
> > [ ID] Interval       Transfer     Bandwidth
> > [  3]  0.0-10.0 sec  9.78 GBytes  8.40 Gbits/sec
> > [  3] 10.0-20.0 sec  8.95 GBytes  7.69 Gbits/sec
> > [  3]  0.0-20.0 sec  18.7 GBytes  8.05 Gbits/sec
>
> Given that iperf exercises the ixgbe driver (ix), network path and TCP,
> I would suggest that your subject is rather misleading ;)
>
> > the card has a 3 meter twinax cable from cisco connected to it, going
> > through a fujitsu switch. We have tweaked various networking, and kernel
> > sysctls, however from a sftp and nfs session i cant get better then
> 100MBs
> > from a zpool with 8 mirrored vdevs. We also have an identical box that
> will
> > get 1.4Gbs with a 1 meter cisco twinax cables that writes 2.4Gbs compared
> > to reads only 1.4Gbs...
>
> I take it the RTT between both hosts is very low i.e. sub 1ms?
>
> > does anyone have an idea of what the bottle neck could be?? This is a
> > shared storage array with dual LSI controllers connected to 32 drives via
> > an enclosure, local dd and other tests show the zpool performs quite
> well.
> > however as soon as we introduce any type of protocol, sftp, samba, nfs
> > performance plummets. Im quite puzzled and have run out of ideas. so now
> > curiousity has me........ its loading the ix driver and working but not
> up
> > to speed,
>
> ssh (and sftp by extension) aren't often tuned for high speed operation.
> Are you running with the HPN patch applied or a new enough FreeBSD that
> has the patch included? Samba and NFS are both likely to need tuning for
> multi-Gbps operation.
>

Running 9-STABLE as of 3 days ago, what are you referring to s i can
validate i dont need to apply it

as for tuning for NFS/SAMBA sambas configured with AIO, and sendfile, and
there so much information
on tuninig these things that its a bit hard to decipher whats right and not
right


>
> Cheers,
> Lawrence
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>