Re: FreeBSD TCP (with iperf3) comparison with Linux

> On 29. Jun 2023, at 11:51, Murali Krishnamurthy <muralik1@vmware.com> wrote:
> 
> Hello FreeBSD Transport experts,
> We are evaluating performance of FreeBSD 13 VM on ESX hypervisor in long RTT setup and happened to compare the performance with Linux VM with same hypervisor.
> We see a substantially better performance with Linux getting close to the BDP limit, whereas BSD 13 not filling up the pipe enough.
> We are trying to figure out what could lead to such a huge difference and feel we could be missing something here.
> Could you please help us to know if there is a way to make it perform better?
> Setup details:
> We have 2 ESX hypervisors where 2 VMs (one FreeBSD 13 and one Ubuntu 23.04/Linux kernel 6.2) were launched on each hypervisor.
> Then we ran iperf between,
>   • BSD 13 <-> BSD 13
>   • Ubuntu <-> Ubuntu
> Even though the network environment were same in both cases, we see Ubuntu performing much better.
> Below are connection parameters:
> Socket buffer: 16MB
> TCP CC Algo: Cubic. We used this as this is suitable for Long Fat Networks.
> Ping RTT:  100 ms between the two end points.
> We kept all other parameters to default on both Linux and BSD.
> BDP for 16MB Socket buffer: 16 MB * (1000 ms * 100ms latency) * 8 bits/ 1024 = 1.25 Gbps
> Ubuntu consistently hits around 1 Gbps Bitrate almost reaching the BDP limit.
> FreeBSD 13 shows a Bit rate between the range of 300-600 Mbps only. So it seems to be doing half as good as Linux.
> For lower socket buffer of 4MB, both FreeBSD and Linux seem to do same and able to meet BDP of 300 Mbps consistently.
> Larger socket buffer seems to have an issue.
> Please let us know if there are ways to fine tune the system parameters to make BSD perform better.
> Or any other suggestions/queries welcome.
> Regards
> Murali
Hi Murali,

I'll bring this up on the biweekly transport call.

Best regards
Michael