Re: 100Gb performance

On Thu, Jun 19, 2025 at 2:34 PM Olivier Cochard-Labbé
<olivier@freebsd.org> wrote:
>
>
>
> On Thu, Jun 19, 2025 at 4:31 PM Rick Macklem <rick.macklem@gmail.com> wrote:
>
>>
>> There is the "nconnect" mount option. It might help here.
>>
>
> Interesting!
>
> Let’s try:
>
> Server side:
> ```
> mkdir /tmp/nfs
> mount -t tmpfs tmpfs /tmp/nfs
> chmod 777 /tmp/nfs/
> cat > /etc/exports <<EOF
> V4: /tmp
> /tmp/nfs -network 1.1.1.0/24
> EOF
> sysrc nfs_server_enable=YES
> sysrc nfsv4_server_enable=YES
> sysrc nfsv4_server_only=YES
> service nfsd start
> ```
>
> Client side:
> ```
> mkdir /tmp/nfs
> sysrc nfs_client_enable=YES
> service nfsclient start
> ```
>
> Now testing standard speed:
> ```
> # mount -t nfs -o noatime,nfsv4 1.1.1.30:/nfs /tmp/nfs/
> # netstat -an -f inet -p tcp | grep 2049 | wc -l
>        1
> # dd if=/dev/zero of=/tmp/nfs/test bs=1G count=10
> 10+0 records in
> 10+0 records out
> 10737418240 bytes transferred in 8.526794 secs (1259256159 bytes/sec)
> # rm /tmp/nfs/test
> # umount /tmp/nfs
> ```
>
> And with nconnect=16:
> ```
> # mount -t nfs -o noatime,nfsv4,nconnect=16 1.1.1.30:/nfs /tmp/nfs/
> # dd if=/dev/zero of=/tmp/nfs/test bs=1G count=10
> 10+0 records in
> 10+0 records out
> 10737418240 bytes transferred in 8.633871 secs (1243638980 bytes/sec)
> # rm /tmp/nfs/test
> # netstat -an -f inet -p tcp | grep 2049 | wc -l
>       16
> ```
>
> => No difference here, but 16 output queues were correctly used with nconnect=16.
> How is load-sharing done with NFS nconnect ?
round robin

> I’ve tested with benchmarks/fio using parallel jobs and I don’t see any improvement too.
And what NIC speed are you testing? (10Gbps or 100Gbps?)
(Basically, you appear to be running at what would be 10Gbps wire speed.)

rick

>
> Regards,
> Olivier