NFS over LAGG / lacp poor performance

[Rick Macklem]

Steven Hartland wrote:
> 
> ----- Original Message -----
> From: "Marek Salwerowicz" <marek_sal at wp.pl>
> To: "Steven Hartland" <killing at multiplay.co.uk>; "Gerrit Kühn"
> <gerrit.kuehn at aei.mpg.de>
> Cc: <freebsd-net at freebsd.org>
> Sent: Friday, April 25, 2014 2:06 PM
> Subject: Re: NFS over LAGG / lacp poor performance
> 
> 
> >W dniu 2014-04-25 14:55, Steven Hartland pisze:
> >> ----- Original Message ----- From: "Marek Salwerowicz"
> >> <marek_sal at wp.pl>
> >>
> >>
> >>> W dniu 2014-04-25 14:01, Gerrit Kühn pisze:
> >>>> Thanks for your input. As far as I understood so far, there
> >>>> should
> >>>> be one
> >>>> igb queue created per cpu core in the system by default (and
> >>>> this is
> >>>> what
> >>>> I see on my system). But my irq rate looks quite high to me (and
> >>>> it is
> >>>> only on one of these queues).
> >>>
> >>>
> >>> My CPU has 8 cores:
> >>>
> >>> http://ark.intel.com/products/75267/Intel-Xeon-Processor-E5-2640-v2-20M-Cache-2_00-GHz
> >>>
> >>>
> >>> So why do I have only 1 queue ?
> >>
> >> What does "sysctl hw.igb.num_queues" report?
> >
> > storage1% sysctl hw.igb.num_queues
> > hw.igb.num_queues: 1
> >>
> >> num_queues does default to 1 for Legacy or MSI so you might be
> >> hitting
> >> that.
> >>
> >> Do you see "Using MSIX interrupts with" in your dmesg?
> > storage% dmesg | grep MSIX
> > igb0: Using MSIX interrupts with 2 vectors
> > igb1: Using MSIX interrupts with 2 vectors
> > igb2: Using MSIX interrupts with 2 vectors
> > igb3: Using MSIX interrupts with 2 vectors
> > igb0: Using MSIX interrupts with 2 vectors
> > igb1: Using MSIX interrupts with 2 vectors
> > igb2: Using MSIX interrupts with 2 vectors
> > igb3: Using MSIX interrupts with 2 vectors
> 
> In that case I believe you've hard coded the number of queues, check
> /boot/loader.conf
> for references to this.
> 
Not really replying to Steve's email, but...

NFS uses a single TCP connection for a mount. I still know nothing
about lagg, but if lagg/lacp requires multiple TCP connections to
spread the load..I'd just switch to using something like ftp, given
you are only moving a few large files.

If you must use NFS, then to get multiple TCP connections, you'll
need to do multiple mounts and then do the file transfers concurrently
over the different mounts.

rick

>     Regards
>     Steve
> 
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe at freebsd.org"
>