From nobody Sun Dec 25 16:37:52 2022 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Ng66M6h0mz20k64 for ; Sun, 25 Dec 2022 16:38:03 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic301-22.consmr.mail.gq1.yahoo.com (sonic301-22.consmr.mail.gq1.yahoo.com [98.137.64.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Ng66L4z34z4X5W for ; Sun, 25 Dec 2022 16:38:02 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=gtNjS7xP; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1671986281; bh=o2eqYsnqE3ib3Jxd0JHm6phoNUmIbd8f9PV0LDCWbQA=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=gtNjS7xPNIa3TeAl0Rqos2ewT7xhBMvuLYrnyJRsKFOZhYVbETmKLkWxALFmMLS08ffQ++NGaz0kCEV0LLlulO6FeWNWMJ/By/+e/R416xRzkNd1/WpNJT34umAG23X5c3MdKFvz1wxkKMZGghdV7ajOqdd4u6a/JcaKWrMBkFEukV8fFjdDMGAEjaaTn53jfEKudzuEF5ghwUX/yY5y9VieCKH8mMdtBnfrC3fn4EBn74Jaznx9DfUisVEoKBAuKzbKTwb1/AW/0i4+u8XnhVviy27l8cAtf+vM+KRHcqbtfuGqsS8HWov3FWw1j+JCtvtZw+E5W+gvKCYRL+J+5w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1671986281; bh=oU0axPVQusdTiiYuyvAD6PvCELptszkPiTI2LOG70N7=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=o5DLsVWKob5RH4GCS89FogwxlFW8j1GksFLYFjD/WjNde5LLDD3cvK52LZTnJpv0tgwlYY+pM+gqVnsmEhr9D7OdQT+YQgsMzMMuZIHPSr/c9x4eT6+ld9HdDlKYcEPcjy1mlPrz5DAqy96//DZCrLuxCiSZTQbKm9ueGEkjuB80VTB5MD975Sd3EdPBOT2V0Ubq8VfzgDicTK7QOY/MrQyIiPzslAZJc4dGfDtZg5+mRkKedEtgC2Z2n9DBSC6SG2/ERHFG7QwaBNc8aJbssXLRnKn6mS0wL34Up57BruTjKodvtFncrFZeeTFxQh7ircA3ObF1KcBGKIooCznFLQ== X-YMail-OSG: JmTvFAMVM1lMHDAHi4ygvpOUH2ZxNjDK3HGOGi0MMlIvL8f2_jvoR880IySEuCa wAiGKVQAnq.B4vX__rpCHLlvlhNlW8aRi3FHsy0EpueYApTz6uAFPo_tr2Fb59ZOTMfE_k0Ef1Ue qQHyt.S.sQa8ax79F6z_bPiUVEY41jBolahLhjkd5aFfV2bC2yzLbqo.jdZnOEH3L.R1ClBepw_N KxFbXofNdzPhKljkwIXTabAP94_c7TMvBddiN2J_vpmQAQ4.ltnFG0JP7zHDNFfWMufgQWShP8Ea hBZtLD6oQMbxpOCpI00xo5qKVAlzrgagZoUKrX0pgZ6nF9cThf_BmUUg9w9LYS9l5UeZHZYfbh7V mX_ceqps.FsEhhb16yKeo1pGzsjGIasGgM0q0.EiQcC7UyFSb2ZvFYfRsrdgPH_.zcRvpJuxOGw0 IKgMERM5hMVBHykxBjg1C1VY2c5eQcNVs2wr3q17k94DWGbKGISnPykr3Z0PFxTRoKkurC.WgoZ2 ZOCGFPPrvWfi10yQKE0PQfC17TAvkdpR8Z3u7GJl8dDGxBsOepuzxNNk6gCxKUNj5hZ6LUM0OY4k 1Z76gJmVOajJ.HXKCfjAKEFEEQQCuF_6jO_.9PXdEAiHqbXeRydTpwHge1ye_iKjcvXNYMdWkDdr iCblfIgBxE04aPjthbY8h2t_.Js2NOOQ7mJEkVr4VjB0Lo.Ncg5WIiDB_PyTK_ZfkRpgEk._HyA. xyuGFW.U.hza_NQjYjsPacXtSktdOb96UcJsQP6pYUObZxb3bJfKILr_nYwnIOlfV_XgO05lmMWs XJLH7XHSHQfHvNJaoQX0KaDpGQclVgr9ktCyD.afV2d40FHFObFilJ9mMw.JzgW6x05nJ5LmxXON 41fKeVhMIJ8kHzpYFkE0HNPQ20KaECy6xIhGvKaznirBM9Ic37HaA07WMBAnZwz.hDQRtDkXzXBa sjiWT2n..HjPVVvJTkRl5Bd.mv.vn1Xcbf5gHQOHZVDayk8gm95a8ziEv60LpAyT3wtb5ijKWi4p 91Z4QwjlcJBQ.IT4oUfCuio09TgQgQDbcR.uvrLH.t.2gIG.3OWai5yYvpHfy6Q6bfNk97jPmhVo LsOqK.Hdl6XlIXJzcQD6tlsj0NvNxvpmoqZdb4dkDg5.k1tBHsjpq8YbDi8swgerJ4rKvhYYnxcQ 7fkQViOIdBZMBKjDmA.wgJXo86lkGj6V_fUCkan0kPVcPl8NZBVby17F8PhKNw_7zYd.d_ZVI9AK Fpr9zxmKwgAv86N3MoMyfKqgfQeUbrKJ3kPkX5r77bU90qpNLKYAuSWjVMq9X6O2Yh_Nshv1ZAdg V3V0WXr4kTijw6vImKGfijVfM2.oddHkzb9qWwLkvq09Eyks27uZU2L40ZijZOhBAXYe6UFamB6J NIZFcP_XPGcax5ieCD5U4wKeoy748bkb8Yvjync5gFvWcVvP3NIP6ZqRJIj4gVqCBVXjqO6UI4Vj yF9ZVrBR6zNaJEpIHrQi_K3mkX9IG3N9FvCkamZPDaN0_tD7e1G44XFyZ9VztCS4gcr_hSsohvzf 5bZ6vithaKquMt6k.pcHXO0kPjo2I5D1.suvf3LPO9eEyEfgVJQcCrO_.cDhgr8ZzT4ieFa__N2C qm3maUWfA24S8LQitsmhMuccZIp_cTwFuAgh.f00W7TsJNLYSY7pC9aI9OKjWL0CkppFCqzLfD2h nlrQ4eR6NuhXVpZD6afDCopdUZp0OOFiIop5VbmQmjsb3RKNm072iLq7f61VCV5rGUz0dFbRrj8F UTFxZ6zDXFrROq.sbQhSWdrfPAR2aK3NYeR_vS_X5sKFJYI9p4qewsWLPKJ.pZUApKiOb4ibQSmo hSIp_LYBqLKd6qKMTXZnBb4zqPhNg4GHCAn0ASG6btn7h3KJNKDIl5252GZeg4SoUU2pyUe2uY68 YojoDw0ZGwAbzcnp0k0orIQnWma_KAluIsYiICLMw.z.9D_JsRMKpsFkqR6fWklEoHFSQlfgMnkl wL9O6Pf9Gzp09gOCtbKNf4NXhF5Kh3qLcuZihXs66EEeV5SCQoFNsMUXH6tVaMm_oaLABtDm0YOt K_YX2AEG5_0izEddksTDRHI4bYtDM4i_DUXCLGJCLe2_DH9P7yMzF14ZTI5nlyzvttHke10GxWd4 lnHZMELxCE3G6ncedSduBOAScyNdZIKQmkYLbfX8dcfJb_lhnDG.24ybUJPzUugWirQGLzsjP903 yZoiHkT9j2O0mh.wMRjARWuohdlejpAOP10KNdDnT57ApTs.cIt6Wu1YwSfz6 X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.gq1.yahoo.com with HTTP; Sun, 25 Dec 2022 16:38:01 +0000 Received: by hermes--production-bf1-5458f64d4-kkg2s (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ae584a7e4e7adf60e9c2eec3aacf3ce4; Sun, 25 Dec 2022 16:37:56 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.300.101.1.3\)) Subject: Re: How to make FreeBSD's kernel boot a RPi4B with modern RPi* firmware Date: Sun, 25 Dec 2022 08:37:52 -0800 References: <9C037D3F-A440-4708-993D-117F313691BB@yahoo.com> <374EC3E5-4CB4-4336-A8B9-7A9CF6151691@yahoo.com> To: freebsd-arm In-Reply-To: <374EC3E5-4CB4-4336-A8B9-7A9CF6151691@yahoo.com> Message-Id: X-Mailer: Apple Mail (2.3731.300.101.1.3) X-Spamd-Result: default: False [-3.48 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.979]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.148:from]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org] X-Rspamd-Queue-Id: 4Ng66L4z34z4X5W X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On Dec 24, 2022, at 20:14, Mark Millard wrote: > On Dec 24, 2022, at 19:15, Mark Millard wrote: >=20 >> I finally ran into EARLY_DRIVER_MODULE, BUS_PASS_RESOURCE, >> BUS_PASS_ORDER_MIDDLE and the like and they allow being >> sure that the brcm,bcm2835-dma related setup has been done >> before any use of it is made, despite the order in the >> Device Tree: use an earlier pass for brcm,bcm2835-dma >> related attach. This avoids the kernel crashing during >> boot. >>=20 >> The example context used below has: serial console with >> USB3 SSD boot media (not requiring a usb_pgood_delay >> setting), booting a stable/13. The RPI4B is a C0T one (no >> 3 GiByte limitation, for example: the PCIe wrapper logic >> has been corrected). >>=20 >> stable/13's source code changes ( similarly for >> releng/13.1 ): >>=20 >> diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >> index cab8639bb607..d8b49acfe332 100644 >> --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c >> +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >> @@ -766,5 +766,6 @@ static driver_t bcm_dma_driver =3D { >>=20 >> static devclass_t bcm_dma_devclass; >>=20 >> -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, bcm_dma_devclass, = 0, 0); >> +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, = bcm_dma_devclass, >> + 0, 0, BUS_PASS_RESOURCE + BUS_PASS_ORDER_MIDDLE); >> MODULE_VERSION(bcm_dma, 1); >>=20 >>=20 >> For reference, a 13S snapshot with my kernel build replacing >> the snapshot's kernel, was booted with: >>=20 >> # strings /boot/msdos/start4.elf | grep VC_BUILD_ID_ >> VC_BUILD_ID_USER: dom >> VC_BUILD_ID_TIME: 11:09:05 >> VC_BUILD_ID_VARIANT: start >> VC_BUILD_ID_TIME: Oct 26 2022 >> VC_BUILD_ID_BRANCH: bcm2711_2 >> VC_BUILD_ID_HOSTNAME: buildbot >> VC_BUILD_ID_PLATFORM: raspberrypi_linux >> VC_BUILD_ID_VERSION: c72ad6b26ff40c91ef776b847436094ee63fabee (clean) >>=20 >> There are new things present that FreeBSD reports >> but ignores, producing messages like: >>=20 >> clk_fixed4: disabled on ofwbus0 >> clk_fixed4: Cannot FDT parameters. >> device_attach: clk_fixed4 attach returned 6 >>=20 >> over and over during part of the boot. It seems to >> retry as it goes and thus produce so many messages. >>=20 >> There was also: >>=20 >> fb0: on simplebus0 >> fb0: changing fb bpp from 0 to 24 >> mbox0: mbox response error >> fb0: bcm2835_mbox_fb_init failed, err=3D5 >> device_attach: fb0 attach returned 6 >>=20 >> genet0 is working. >>=20 >> I've not checked if the microsd card slot can be used. >>=20 >> I used the normal FreeBSD U-Boot since I was not booting >> the NVM3 media that requires extra time (usb_pgood_delay >> would be assigned in my own U-Boot builds). >>=20 >> For reference, I used my typical sort of config.txt : >>=20 >> # more /boot/msdos/config.txt >> [all] >> arm_64bit=3D1 >> dtparam=3Daudio=3Don,i2c_arm=3Don,spi=3Don >> dtoverlay=3Dmmc >> dtoverlay=3Ddisable-bt >> device_tree_address=3D0x4000 >> kernel=3Du-boot.bin >>=20 >> [pi4] >> hdmi_safe=3D1 >> armstub=3Darmstub8-gic.bin >>=20 >> # >> [all] >> # >> # Local addition that avoids USB3 SSD boot failures that look like: >> # uhub_reattach_port: port ? reset failed, error=3DUSB_ERR_TIMEOUT >> # uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling = port ? >> initial_turbo=3D60 >> # U-Boot that has, for example, a built-in usb_pgood_delay assignment >> # for a media specific issue added: >> #kernel=3Du-boot.bin.2022.10.arm64 >> # >> # Local additions: >> enable_uart=3D1 >> uart_2ndstage=3D1 >> dtdebug=3D1 >> disable_commandline_tags=3D1 >> disable_overscan=3D1 >> #gpu_mem_1024=3D32 >> # >> #program_usb_boot_mode=3D1 >> #program_usb_boot_timeout=3D1 >>=20 >> # Old RPi3's/RPi2Bv1.2's may ignore [pi4] and the like. >> # That would make the below inappropriate for such contexts. >> [pi4] >> # Locally avoid hdmi_safe's dislay scaling: >> hdmi_safe=3D0 >> # >> armstub=3Darmstub8-gic.bin >> # >> # Local additions: >> over_voltage=3D6 >> arm_freq=3D2000 >> sdram_freq_min=3D3200 >> force_turbo=3D1 >> # >> #total_mem=3D1024 >> #total_mem=3D991 >> [all] >>=20 >> [pi3]=20 >> armstub=3Darmstub8.bin >> dtoverlay=3Dpwm >> audio_pwm_mode=3D0 >> [all] >>=20 >>=20 >> As for main [so: 14], the devclass_t use is gone, thus >> the source code changes are: >>=20 >> diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >> index 5f9ecb0b7981..83c4c493a66b 100644 >> --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c >> +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c >> @@ -764,5 +764,6 @@ static driver_t bcm_dma_driver =3D { >> sizeof(struct bcm_dma_softc), >> }; >>=20 >> -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0); >> +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0, >> + BUS_PASS_RESOURCE + BUS_PASS_ORDER_MIDDLE); >> MODULE_VERSION(bcm_dma, 1); >>=20 >>=20 >=20 > I should note that the above is not likely to be > the most appropriate in detail. The boot reports: >=20 > # dmesg -a | grep bcm_dma0 > bcm_dma0: mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 > bcm_dma0: cannot allocate interrupt > device_attach: bcm_dma0 attach returned 6 > bcm_dma0: mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 > bcm_dma0: cannot allocate interrupt > device_attach: bcm_dma0 attach returned 6 > bcm_dma0: mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 > bcm_dma0: cannot allocate interrupt > device_attach: bcm_dma0 attach returned 6 > bcm_dma0: mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 > bcm_dma0: cannot allocate interrupt > device_attach: bcm_dma0 attach returned 6 > bcm_dma0: mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >=20 > where that last (working) one has the message > context: >=20 > gic0: mem = 0x40041000-0x40041fff,0x40042000-0x40043fff,0x40044000-0x40045fff,0x400460= 00-0x40047fff irq 30 on simplebus0 > gic0: pn 0x2, arch 0x2, rev 0x1, implementer 0x43b irqs 256 > bcm_dma0: mem 0x7e007000-0x7e007aff irq = 31,32,33,34,35,36,37,38,39,40,41 on simplebus0 >=20 > So something involving BUS_PASS_INTERRUPT or later > (but before, say, BUS_PASS_SUPPORTDEV) may be more > appropriate. Possibly: >=20 > BUS_PASS_INTERRUPT + BUS_PASS_ORDER_LATE >=20 > (so after gic0). >=20 >=20 So, I'm now using . . . (leading whitespace possibly not accurately preserved) stable/13's source code changes are ( similarly for releng/13.1 ): # git -C /usr/13S-src/ diff sys/arm/broadcom/bcm2835/bcm2835_dma.c diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c index cab8639bb607..6d521d6dcace 100644 --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c @@ -766,5 +766,6 @@ static driver_t bcm_dma_driver =3D { =20 static devclass_t bcm_dma_devclass; =20 -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, bcm_dma_devclass, 0, = 0); +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, = bcm_dma_devclass, + 0, 0, BUS_PASS_INTERRUPT + BUS_PASS_ORDER_LATE); MODULE_VERSION(bcm_dma, 1); main's [so: 14's] source code changes are: # git -C /usr/main-src/ diff sys/arm/broadcom/bcm2835/bcm2835_dma.c diff --git a/sys/arm/broadcom/bcm2835/bcm2835_dma.c = b/sys/arm/broadcom/bcm2835/bcm2835_dma.c index 5f9ecb0b7981..d901447df1e9 100644 --- a/sys/arm/broadcom/bcm2835/bcm2835_dma.c +++ b/sys/arm/broadcom/bcm2835/bcm2835_dma.c @@ -764,5 +764,6 @@ static driver_t bcm_dma_driver =3D { sizeof(struct bcm_dma_softc), }; =20 -DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0); +EARLY_DRIVER_MODULE(bcm_dma, simplebus, bcm_dma_driver, 0, 0, + BUS_PASS_INTERRUPT + BUS_PASS_ORDER_LATE); MODULE_VERSION(bcm_dma, 1); =3D=3D=3D Mark Millard marklmi at yahoo.com