Re: FreeBSD 14.0-RELEASE and Raspberry Pi CM4 4GB
- Reply: Mark Millard : "Re: FreeBSD 14.0-RELEASE and Raspberry Pi CM4 4GB"
- In reply to: Mike Karels : "Re: FreeBSD 14.0-RELEASE and Raspberry Pi CM4 4GB"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Sat, 23 Dec 2023 01:28:04 UTC
On Dec 22, 2023, at 14:48, Mike Karels <mike@karels.net> wrote: On 22 Dec 2023, at 16:14, Steve Bernacki wrote: > >> Hi Mike, >> >> Indeed, I'm getting a lot of retransmits: >> >> [ 5] local 172.16.200.2 port 55551 connected to 172.16.200.182 port 5201 >> [ ID] Interval Transfer Bitrate Retr Cwnd >> [ 5] 0.00-1.00 sec 36.2 MBytes 304 Mbits/sec 60 9.98 KBytes >> [ 5] 1.00-2.00 sec 35.7 MBytes 300 Mbits/sec 143 111 KBytes >> [ 5] 2.00-3.00 sec 34.9 MBytes 293 Mbits/sec 141 7.13 KBytes >> [ 5] 3.00-4.00 sec 33.9 MBytes 284 Mbits/sec 198 99.5 KBytes >> [ 5] 4.00-5.00 sec 34.9 MBytes 292 Mbits/sec 167 1.43 KBytes >> [ 5] 5.00-6.00 sec 34.2 MBytes 287 Mbits/sec 221 2.85 KBytes >> [ 5] 6.00-7.00 sec 34.1 MBytes 286 Mbits/sec 169 100 KBytes >> [ 5] 7.00-8.00 sec 35.2 MBytes 295 Mbits/sec 159 7.13 KBytes >> [ 5] 8.00-9.00 sec 34.3 MBytes 287 Mbits/sec 138 4.28 KBytes >> [ 5] 9.00-10.00 sec 33.3 MBytes 279 Mbits/sec 182 2.85 KBytes >> - - - - - - - - - - - - - - - - - - - - - - - - - >> [ ID] Interval Transfer Bitrate Retr >> [ 5] 0.00-10.00 sec 347 MBytes 291 Mbits/sec 1578 sender >> [ 5] 0.00-10.00 sec 346 MBytes 291 Mbits/sec receiver >> >> Thanks, >> Steve > > One other question: are you running powerd? I booted without it, and my > throughput dropped to 600-640 Mb/s. Repeating the test, retransmissions > went down but throughput was about the same. Note, the RPi 4, and probably > the CM 4, boots at a lower clock frequency by default, and powerd raises it > under load. I'm running powerd with -M 1800, overclocking a little. I explore here fixed frequencies: 2000 MHz, 600 MHz, 1500 MHz, 1800 MHz (no powerd use) Based on: # uname -apKU FreeBSD generic 14.0-STABLE FreeBSD 14.0-STABLE #0 stable/14-n266002-2ef9079ece5a: Sat Dec 16 08:49:23 UTC 2023 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64 aarch6 # more /boot/efi/config.txt [all] arm_64bit=1 dtparam=audio=on,i2c_arm=on,spi=on dtoverlay=mmc dtoverlay=disable-bt device_tree_address=0x4000 kernel=u-boot.bin [pi4] hdmi_safe=1 armstub=armstub8-gic.bin # over_voltage=6 sdram_freq_min=3200 arm_freq_min=2000 force_turbo=1 # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels dev.cpu.0.freq dev.bcm2835_cpufreq.0.freq_settings: 2000/-1 dev.cpu.0.freq_levels: 2000/-1 dev.cpu.0.freq: 2000 # iperf3 -c 192.168.1.157 Connecting to host 192.168.1.157, port 5201 [ 5] local 192.168.1.159 port 52424 connected to 192.168.1.157 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 113 MBytes 948 Mbits/sec 243 328 KBytes [ 5] 1.00-2.00 sec 112 MBytes 941 Mbits/sec 150 18.5 KBytes [ 5] 2.00-3.00 sec 112 MBytes 942 Mbits/sec 149 173 KBytes [ 5] 3.00-4.00 sec 112 MBytes 942 Mbits/sec 150 456 KBytes [ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec 159 456 KBytes [ 5] 5.00-6.00 sec 112 MBytes 941 Mbits/sec 160 538 KBytes [ 5] 6.00-7.00 sec 112 MBytes 941 Mbits/sec 143 1.43 KBytes [ 5] 7.00-8.00 sec 112 MBytes 942 Mbits/sec 215 167 KBytes [ 5] 8.00-9.00 sec 112 MBytes 941 Mbits/sec 194 580 KBytes [ 5] 9.00-10.00 sec 112 MBytes 941 Mbits/sec 157 552 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.10 GBytes 942 Mbits/sec 1720 sender [ 5] 0.00-10.01 sec 1.10 GBytes 941 Mbits/sec receiver iperf Done. Note: The amd64 system running main [so: 15] and the RPi4B are on the same ethernet switch. With the 4 overclocking lines in config.txt commented out : # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels dev.cpu.0.freq dev.bcm2835_cpufreq.0.freq_settings: 1500/-1 600/-1 dev.cpu.0.freq_levels: 1500/-1 600/-1 dev.cpu.0.freq: 600 Note: the default context lacks 1800 (based on the RPi* firmware vintage in the snapshot). Later I show having 1800 instead of 1500. # iperf3 -c 192.168.1.157 Connecting to host 192.168.1.157, port 5201 [ 5] local 192.168.1.159 port 42060 connected to 192.168.1.157 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 70.8 MBytes 594 Mbits/sec 18 195 KBytes [ 5] 1.00-2.00 sec 73.8 MBytes 619 Mbits/sec 8 293 KBytes [ 5] 2.00-3.00 sec 73.6 MBytes 618 Mbits/sec 19 250 KBytes [ 5] 3.00-4.00 sec 73.6 MBytes 618 Mbits/sec 9 366 KBytes [ 5] 4.00-5.00 sec 73.3 MBytes 615 Mbits/sec 9 447 KBytes [ 5] 5.00-6.00 sec 73.3 MBytes 615 Mbits/sec 16 303 KBytes [ 5] 6.00-7.00 sec 73.2 MBytes 614 Mbits/sec 0 455 KBytes [ 5] 7.00-8.00 sec 73.6 MBytes 618 Mbits/sec 1 328 KBytes [ 5] 8.00-9.00 sec 73.5 MBytes 616 Mbits/sec 16 246 KBytes [ 5] 9.00-10.00 sec 73.3 MBytes 615 Mbits/sec 0 435 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 732 MBytes 614 Mbits/sec 96 sender [ 5] 0.00-10.01 sec 732 MBytes 613 Mbits/sec receiver iperf Done. Assigning 1500: # sysctl dev.cpu.0.freq=1500 dev.cpu.0.freq: 600 -> 1500 # sysctl dev.cpu.0.freq=1500 dev.cpu.0.freq: 600 -> 1500 root@generic:~ # iperf3 -c 192.168.1.157 Connecting to host 192.168.1.157, port 5201 [ 5] local 192.168.1.159 port 28904 connected to 192.168.1.157 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 113 MBytes 949 Mbits/sec 4 472 KBytes [ 5] 1.00-2.00 sec 112 MBytes 941 Mbits/sec 6 464 KBytes [ 5] 2.00-3.00 sec 112 MBytes 941 Mbits/sec 5 452 KBytes [ 5] 3.00-4.00 sec 112 MBytes 941 Mbits/sec 3 443 KBytes [ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec 4 421 KBytes [ 5] 5.00-6.00 sec 112 MBytes 941 Mbits/sec 4 397 KBytes [ 5] 6.00-7.00 sec 112 MBytes 942 Mbits/sec 3 378 KBytes [ 5] 7.00-8.00 sec 112 MBytes 941 Mbits/sec 5 355 KBytes [ 5] 8.00-9.00 sec 112 MBytes 941 Mbits/sec 2 476 KBytes [ 5] 9.00-10.00 sec 112 MBytes 941 Mbits/sec 5 446 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.10 GBytes 942 Mbits/sec 41 sender [ 5] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec receiver Adding arm_boost=1 to config.txt in order to have 1800 instead of 1500 (needed due to the RPi* firmware vintage in FreeBSD snapshots): # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels dev.cpu.0.freq dev.bcm2835_cpufreq.0.freq_settings: 1800/-1 600/-1 dev.cpu.0.freq_levels: 1800/-1 600/-1 dev.cpu.0.freq: 600 # sysctl dev.cpu.0.freq=1800 dev.cpu.0.freq: 600 -> 1800 # iperf3 -c 192.168.1.157 Connecting to host 192.168.1.157, port 5201 [ 5] local 192.168.1.159 port 27499 connected to 192.168.1.157 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 114 MBytes 952 Mbits/sec 169 104 KBytes [ 5] 1.00-2.00 sec 112 MBytes 941 Mbits/sec 150 320 KBytes [ 5] 2.00-3.00 sec 112 MBytes 942 Mbits/sec 157 52.8 KBytes [ 5] 3.00-4.00 sec 112 MBytes 941 Mbits/sec 143 87.0 KBytes [ 5] 4.00-5.00 sec 112 MBytes 941 Mbits/sec 143 121 KBytes [ 5] 5.00-6.00 sec 112 MBytes 941 Mbits/sec 159 104 KBytes [ 5] 6.00-7.00 sec 112 MBytes 941 Mbits/sec 138 238 KBytes [ 5] 7.00-8.00 sec 112 MBytes 941 Mbits/sec 152 276 KBytes [ 5] 8.00-9.00 sec 112 MBytes 941 Mbits/sec 145 115 KBytes [ 5] 9.00-10.00 sec 112 MBytes 941 Mbits/sec 162 283 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.10 GBytes 943 Mbits/sec 1518 sender [ 5] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec receiver iperf Done. From this it appears that the Retr counts do not seem to make much of a difference to the Bitrate's achieved. But the arm frequency does if 600 is involved. My understanding is that arm_boost=1 was later made the default in later vintages of the rpi* firmware. arm_boots only causes 1800 for Rev 1.4+ . Pi 400's have 1800 available by default, at least for modern enough RPi* firmware. https://www.raspberrypi.com/documentation/computers/config_txt.html is not necessarily accurate for the older RPi* firmware that FreeBSD uses in its snapshots/releases. > IIRC > the standard clock is 1500 for the RPi 4. But the throughput is about the > same using the standard clock with powerd. > > Mike > >> On 12/22/2023 9:23 AM, Mike Karels wrote: >>> On 22 Dec 2023, at 6:20, Steve Bernacki wrote: >>> >>>> I recently purchased a RPI CM4 with 4GB and 32GB eMMC to replace my aging FreeBSD firewall. I managed to install FreeBSD 14.0-RELEASE-p3 on it, and both Ethernet devices (genet0 and ue0) were properly identified. However, network throughput on my gigabit network is pretty bad; iperf3 reports a maximum transfer speed of 291 Mbits/sec. Flashing OpenWRT on the same hardware using the same ethernet port, I'm able to achieve 923 Mbits/sec. >>>> >>>> Does anyone have any suggestions on how to improve throughput under FreeBSD? >>>> >>>> Thank you >>>> Steve >>> I just tested with an RPi4 (4 GB) and 14.0 using iperf3. It looks like I'm getting >>> a rather variable number of retransmissions. On my first run (client on RPi 4), >>> I got 460 Mb/s with a lot of retransmissions, but the next couple of runs, including >>> one receiving, I got about 940 Mb even with some retransmissions. The peers were >>> fairly fast FreeBSD 13.2 and 15-current systems. Are you seeing retransmissions? >>> >>> I'll try to look into this, but I'm not sure when I'll get to it. >>> >>> Mike > === Mark Millard marklmi at yahoo.com