Re: Rock64 configuration fails to boot for main 22c4ab6cb015 but worked for main 06bd74e1e39c (Nov 21): e.MMC mishandled?

On Thu, Dec 9, 2021 at 11:54 PM Mark Millard <marklmi@yahoo.com> wrote:
>
> On 2021-Dec-9, at 08:01, Kornel Dulęba <mindal@semihalf.com> wrote:
>
> > Hi,
> >
> > On Thu, Dec 9, 2021 at 2:04 PM Mark Millard via freebsd-arm
> > <freebsd-arm@freebsd.org> wrote:
> >>
> >> Hello again.
> >>
> >> On 2021-Dec-9, at 00:43, Mark Millard <marklmi@yahoo.com> wrote:
> >>
> >>> On 2021-Dec-8, at 23:19, Emmanuel Vadot <manu@bidouilliste.com> wrote:
> >>>
> >>>> Hi Mark,
> >>>
> >>> Hello.
> >>>
> >>>> On Wed, 8 Dec 2021 20:36:20 -0800
> >>>> Mark Millard via freebsd-current <freebsd-current@freebsd.org> wrote:
> >>>>
> >>>>> [ Note: wma@FreeBSD.org is only a guess, based on:
> >>>>> https://lists.freebsd.org/archives/dev-commits-src-main/2021-December/001931.html ]
> >>>>>
> >>>>> Attempting to update to:
> >>>>>
> >>>>> main-n251456-22c4ab6cb015-dirty: Tue Dec  7 19:38:53 PST 2021
> >>>>>
> >>>>> resulted in boot failure (showing some boot -v output):
> >>>>>
> >>>>> . . .
> >>>>> mmc0: Probing bus
> >>>>> . . .
> >>>>> mmc0: SD probe: failed
> >>>>> mmc0: MMC probe: OK (OCR: 0x40ff8080)
> >>>>> mmc0: Current OCR: 0x00ff8080
> >>>>> mmc0: Probing cards
> >>>>> mmc0: New card detected (CID 150100444a4e423452079f43b2ae6313)
> >>>>> mmc0: New card detected (CSD d02701320f5903fff6dbffef8e40400d)
> >>>>> mmc0: Card at relative address 0x0002 added:
> >>>>> mmc0:  card: MMCHC DJNB4R 0.7 SN REPLACED MFG 06/2016 by 21 0x0000
> >>>>> mmc0:  quirks: 0
> >>>>> mmc0:  bus: 8bit, 200MHz (HS400 with enhanced strobe timing)
> >>>>> mmc0:  memory: 244277248 blocks, erase sector 1024 blocks
> >>>>> mmc0: setting transfer rate to 150.000MHz (HS200 timing)
> >>>>> mmcsd0: taking advantage of TRIM
> >>>>> mmcsd0: cache size 65536KB
> >>>>> mmcsd0: 125GB <MMCHC DJNB4R 0.7 SN REPLACED MFG 06/2016 by 21 0x0000> at mmc0 150.0MHz/8bit/1016-block
> >>>>> mmcsd0boot0: 4MB partition 1 at mmcsd0
> >>>>> mmcsd0boot1: 4MB partition 2 at mmcsd0
> >>>>> mmcsd0rpmb: 4MB partition 3 at mmcsd0
> >>>>> . . .
> >>>>> Release APs...done
> >>>>> regulator: shutting down unused regulators
> >>>>> GEOM: new disk mmcsd0
> >>>>> regulator: shutting down vcc_sd... GEOM: new disk mmcsd0boot0
> >>>>> busy
> >>>>> GEOM: new disk mmcsd0boot1
> >>>>> Trying to mount root from ufs:/dev/gpt/Rock64root []...
> >>>>> Unresolved linked clock found: hdmi_phy
> >>>>> Unresolved linked clock found: usb480m_phy
> >>>>> mmcsd0: Error indicated: 4 Failed
> >>>>>
> >>>>> Note the the above line. It seems to be unique to
> >>>>> the failure. Continuing the output . . .
> >>>>>
> >>>>> uhub2: 1 port with 1 removable, self powered
> >>>>> uhub1: 2 ports with 2 removable, self powered
> >>>>> uhub0: 1 port with 1 removable, self powered
> >>>>> uhub3: 1 port with 1 removable, self powered
> >>>>> ugen4.2: <vendor 0x04e8 product 0x4001> at usbus4
> >>>>> umass0 on uhub1
> >>>>> umass0: <vendor 0x04e8 product 0x4001, class 0/0, rev 3.20/1.00, addr 1> on usbus4
> >>>>> umass0:  SCSI over Bulk-Only; quirks = 0x0000
> >>>>> umass0:0:0: Attached to scbus0
> >>>>> pass0 at umass-sim0 bus 0 scbus0 target 0 lun 0
> >>>>> pass0: <Samsung PSSD T7 Touch 0> Fixed Direct Access SPC-4 SCSI device
> >>>>> pass0: Serial Number REPLACED
> >>>>> pass0: 400.000MB/s transfers
> >>>>> da0 at umass-sim0 bus 0 scbus0 target 0 lun 0
> >>>>> da0: <Samsung PSSD T7 Touch 0> Fixed Direct Access SPC-4 SCSI device
> >>>>> da0: Serial Number REPLACED
> >>>>> da0: 400.000MB/s transfers
> >>>>> da0: 953869MB (1953525168 512 byte sectors)
> >>>>> da0: quirks=0x2<NO_6_BYTE>
> >>>>> da0: Delete methods: <NONE(*),ZERO>
> >>>>>
> >>>>> Nothing more after that.
> >>>>>
> >>>>> An older kernel (1400042) that happened to be available boots
> >>>>> the same configuration when used instead (same world) . . .
> >>>>>
> >>>>> main-n250903-06bd74e1e39c-dirty: Sun Nov 21 23:02:57 PST 2021 got:
> >>>>>
> >>>>> mmc0: Probing bus
> >>>>> . . .
> >>>>> mmc0: SD probe: failed
> >>>>> mmc0: MMC probe: OK (OCR: 0x40ff8080)
> >>>>> mmc0: Current OCR: 0x00ff8080
> >>>>> mmc0: Probing cards
> >>>>> mmc0: New card detected (CID 150100444a4e423452079f43b2ae6313)
> >>>>> mmc0: New card detected (CSD d02701320f5903fff6dbffef8e40400d)
> >>>>> mmc0: Card at relative address 0x0002 added:
> >>>>> mmc0:  card: MMCHC DJNB4R 0.7 SN REPLACED MFG 06/2016 by 21 0x0000
> >>>>> mmc0:  quirks: 0
> >>>>> mmc0:  bus: 8bit, 200MHz (HS400 with enhanced strobe timing)
> >>>>> mmc0:  memory: 244277248 blocks, erase sector 1024 blocks
> >>>>> mmc0: setting transfer rate to 52.000MHz (high speed timing)
> >>>>>
> >>>>> Note the lack of trying "150.000MHz (HS200 timing)".  Continuing
> >>>>> the output . . .
> >>>>>
> >>>>> mmc0: setting bus width to 8 bits high speed timing
> >>>>> mmcsd0: taking advantage of TRIM
> >>>>> mmcsd0: cache size 65536KB
> >>>>> mmcsd0: 125GB <MMCHC DJNB4R 0.7 SN REPLACED MFG 06/2016 by 21 0x0000> at mmc0 52.0MHz/8bit/1016-block
> >>>>> mmcsd0boot0: 4MB partition 1 at mmcsd0
> >>>>> mmcsd0boot1: 4MB partition 2 at mmcsd0
> >>>>> mmcsd0rpmb: 4MB partition 3 at mmcsd0
> >>>>>
> >>>>> Note: The media is actually an e.MMC . Continuing the output . . .
> >>>>>
> >>>>> . . .
> >>>>> Release APs...done
> >>>>> regulator: shutting down unused regulators
> >>>>> GEOM: new disk mmcsd0
> >>>>> regulator: shutting down vcc_sd... Trying to mount root from ufs:/dev/gpt/Rock64root []...
> >>>>> GEOM: new disk mmcsd0boot0
> >>>>> busy
> >>>>> GEOM: new disk mmcsd0boot1
> >>>>> Unresolved linked clock found: hdmi_phy
> >>>>> Unresolved linked clock found: usb480m_phy
> >>>>> Root mount waiting for: usbus1 usbus2 usbus3 usbus4 CAM
> >>>>> uhub1: 1 port with 1 removable, self powered
> >>>>> uhub0: 2 ports with 2 removable, self powered
> >>>>> uhub3: 1 port with 1 removable, self powered
> >>>>> uhub2: 1 port with 1 removable, self powered
> >>>>> ugen4.2: <vendor 0x04e8 product 0x4001> at usbus4
> >>>>> umass0 on uhub0
> >>>>> umass0: <vendor 0x04e8 product 0x4001, class 0/0, rev 3.20/1.00, addr 1> on usbus4
> >>>>> umass0:  SCSI over Bulk-Only; quirks = 0x0000
> >>>>> umass0:0:0: Attached to scbus0
> >>>>> Root mount waiting for: CAM
> >>>>> Root mount waiting for: CAM
> >>>>> Root mount waiting for: CAM
> >>>>> Root mount waiting for: CAM
> >>>>> Root mount waiting for: CAM
> >>>>> Root mount waiting for: CAM
> >>>>> Root mount waiting for: CAM
> >>>>> Root mount waiting for: CAM
> >>>>> Root mount waiting for: CAM
> >>>>> GEOM: new disk da0
> >>>>> pass0 at umass-sim0 bus 0 scbus0 target 0 lun 0
> >>>>> pass0: <Samsung PSSD T7 Touch 0> Fixed Direct Access SPC-4 SCSI device
> >>>>> pass0: Serial Number REPLACED
> >>>>> pass0: 400.000MB/s transfers
> >>>>> da0 at umass-sim0 bus 0 scbus0 target 0 lun 0
> >>>>> da0: <Samsung PSSD T7 Touch 0> Fixed Direct Access SPC-4 SCSI device
> >>>>> da0: Serial Number REPLACED
> >>>>> da0: 400.000MB/s transfers
> >>>>> da0: 953869MB (1953525168 512 byte sectors)
> >>>>> da0: quirks=0x2<NO_6_BYTE>
> >>>>> da0: Delete methods: <NONE(*),ZERO>
> >>>>> random: unblocking device.
> >>>>> Warning: bad time from time-of-day clock, system time will not be set accurately
> >>>>> Dual Console: Serial Primary, Video Secondary
> >>>>> start_init: trying /sbin/init
> >>>>> . . .
> >>>>>
> >>>>> (I'll stop with that.)
> >>>>>
> >>>>> So I end up with a 1400042 kernel and a 1400043 world in order to
> >>>>> boot.
> >>>>>
> >>>>> The e.MMC has only:
> >>>>>
> >>>>> # ls -FTld *
> >>>>> -r--r--r--   1 root  wheel   6170 Feb  1 04:48:34 2020 COPYRIGHT
> >>>>> drwxr-xr-x  23 root  wheel   1536 Dec  8 20:18:34 2021 boot/
> >>>>> drwxr-xr-x   2 root  wheel    512 Apr 26 14:39:22 2020 etc/
> >>>>> drwx------   2 root  wheel  33280 Nov 27 09:46:08 2019 lost+found/
> >>>>>
> >>>>> where the etc/ has only:
> >>>>>
> >>>>> # find etc/ -print
> >>>>> etc/
> >>>>> etc/hostid
> >>>>>
> >>>>> World comes from the USB3 SSD that is attached but the kernel
> >>>>> comes from the e.MMC instead. (The kernel can deal with the
> >>>>> USB3 SSD just fine, unlike the U-Boot that is involved.)
> >>>>>
> >>>>> ===
> >>>>> Mark Millard
> >>>>> marklmi at yahoo.com
> >>>>> ( dsl-only.net went
> >>>>> away in early 2018-Mar)
> >>>>>
> >>>>>
> >>>>
> >>>> Could you try reverting
> >>>> 8661e085fb953855dbc7059f21a64a05ae61b22c "mmc: Fix HS200/HS400
> >>>> capability check" and let me know ?
> >>>
> >>> I'm in the middle of something on the systems so it may be a while
> >>> before I do that. (I think it will be my first individual revert
> >>> of some specific old change via the git context. Hmm.)
> >>>
> >>> Also, I do not know enough to tell the difference between:
> >>>
> >>> that test being wrong
> >>> vs.
> >>> mishandling of the combination (presuming it is supposed to be valid)
> >>>
> >>> So I may end up just reporting if it reverts to the old settings
> >>> being in use vs. not.
> >>>
> >>> But . . .
> >>>
> >>> I've an old Odroid C2 with an old NetBSD 9.0_STABLE (GENERIC64) on
> >>> it that is on the same type of e.MMC device and the e.MMC is used
> >>> to boot. That old NetBSD reports for the ODroid C2 during booting:
> >>>
> >>> [   1.8295810] ld1 at sdmmc1: <0x15:0x0100:DJNB4R:0x00:0xddebe217:0x000>
> >>> [   1.8295810] ld1: 116 GB, 15205 cyl, 255 head, 63 sec, 512 bytes/sect x 244277248 sectors
> >>> [   1.8439125] ld1: 8-bit width, HS200, 64 MB cache, 100.000 MHz
> >>>
> >>> So it appears that some form of HS200 is a possibility as far
> >>> as the e.MMC device is concerned. (I make no claims about related
> >>> Rock64 vs. ODroid hardware capability differences --or FreeBSD's
> >>> intent for the Rock64 in this area.)
> >>>
> >>
> >> So, I tried NetBSD 9.2_STABLE on the Rock64 with a e.MMC as the
> >> boot media: it does not use HS200 mode (so it uses 52 MHz, not
> >> 100 MHz).
> >>
> >> [   1.8672737] ld1 at sdmmc1: <0x15:0x0100:DJNB4R:0x00:0x9f43b223:0x000>
> >> [   1.8773160] ld1: 116 GB, 15205 cyl, 255 head, 63 sec, 512 bytes/sect x 244277248 sectors
> >> [   1.8773160] ld1: 8-bit width, 64 MB cache, 52.000 MHz
> >>
> >> Not that I know any details about why, but it does suggest that
> >> the issue may be Rock64 specific, given what NetBSD 9.2_STABLE
> >> does with HS200 on the ODroid C2 . (I updated the C2's NetBSD
> >> vintage to match the Rock64 experiment so comparison/contrast
> >> results would be more reasonable.)
> >
> > I don't think de9c000cedfe has anything to do with it as it only
> > modifies the Freescale sdhci driver.
> > The Rock64 MMC controller doesn't use that code.
> >
> > In 8661e085fb95 I fixed the HS200/HS400 capability detection primarily
> > for DT based controllers.
> > Rock64 controller declares HS200 support with "mmc-hs200-1_8v;"
> > property in the eMMC node in rk3328-rock64.dts.
> > I guess that since it doesn't work the quickest WA for this would be
> > to add SDHCI_QUIRK_BROKEN_MMC_HS200 to sc->quirks in sdhci_fdt.c.
>
> Well, I've tried Armbian 21.08 (Linux 5.10.60-rockchip64) and its first
> boot reports the sequence ended up using HS200 at 150 MHz:
>
> # dmesg | grep mmc
> [    3.195642] vcc18_emmc: supplied by vcc_io
> [    3.227180] dwmmc_rockchip ff520000.mmc: IDMAC supports 32-bit address mode.
> [    3.227187] dwmmc_rockchip ff500000.mmc: IDMAC supports 32-bit address mode.
> [    3.227225] dwmmc_rockchip ff520000.mmc: Using internal DMA controller.
> [    3.227234] dwmmc_rockchip ff500000.mmc: Using internal DMA controller.
> [    3.227244] dwmmc_rockchip ff520000.mmc: Version ID is 270a
> [    3.227259] dwmmc_rockchip ff500000.mmc: Version ID is 270a
> [    3.227342] dwmmc_rockchip ff520000.mmc: DW MMC controller at irq 42,32 bit host data width,256 deep fifo
> [    3.227390] dwmmc_rockchip ff500000.mmc: DW MMC controller at irq 41,32 bit host data width,256 deep fifo
> [    3.229762] mmc_host mmc0: card is non-removable.
> [    3.241627] mmc_host mmc1: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0)
> [    3.241860] mmc_host mmc0: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0)
>
> Note the below 3 lines:

>
> [    3.327640] mmc_host mmc0: Bus speed (slot 0) = 150000000Hz (slot req 150000000Hz, actual 150000000HZ div = 0)
> [    3.730166] dwmmc_rockchip ff520000.mmc: Successfully tuned phase to 245
> [    3.730397] mmc0: new HS200 MMC card at address 0001
>
> Note the "tuned phase to 245" as part of that.

Yep, it looks like in Linux they're doing some custom tuning logic
specific to this controller.
FreeBSD only executes generic tuning code, which apparently is not enough.

>
> [    3.732538] mmcblk0: mmc0:0001 DJNB4R 116 GiB
> [    3.733510] mmcblk0boot0: mmc0:0001 DJNB4R partition 1 4.00 MiB
> [    3.734513] mmcblk0boot1: mmc0:0001 DJNB4R partition 2 4.00 MiB
> [    3.734917] mmcblk0rpmb: mmc0:0001 DJNB4R partition 3 4.00 MiB, chardev (243:0)
> [    3.746005]  mmcblk0: p1
> [    4.880861] EXT4-fs (mmcblk0p1): mounted filesystem with writeback data mode. Opts: (null)
> [    6.686795] EXT4-fs (mmcblk0p1): re-mounted. Opts: commit=600,errors=remount-ro
> [   12.767622] EXT4-fs (mmcblk0p1): resizing filesystem from 479232 to 30224384 blocks
> [   22.791358] EXT4-fs (mmcblk0p1): resized to 16252928 blocks
> [   31.531320] EXT4-fs (mmcblk0p1): resized filesystem to 30224384
>
> So, as far as I can tell, if FreeBSD wants to support HS200 at 150 MHz
> on the Rock64, it can be done, voltage changing and tuning apparently
> involved.
>
> That is not to say that any FreeBSD developer wants to be supporting such.
> Sticking to 52 MHz and possibly 3V for the Rock 64 eMMC use would again
> make things operational.

Yes, imho marking HS200 in this controller as broken is the right
choice, unless someone(TM) writes the missing code.

>
> I'll note that Armbian's U-Boot reports itself as:
>
> U-Boot 2020.10-armbian (Aug 08 2021 - 18:02:43 +0200)
>
> I'll also note that rebooting swapped which was mmc0 vs. mmc1:
>
> # dmesg | grep mmc
> [    3.198267] vcc18_emmc: supplied by vcc_io
> [    3.229498] dwmmc_rockchip ff500000.mmc: IDMAC supports 32-bit address mode.
> [    3.229547] dwmmc_rockchip ff500000.mmc: Using internal DMA controller.
> [    3.229566] dwmmc_rockchip ff500000.mmc: Version ID is 270a
> [    3.229665] dwmmc_rockchip ff500000.mmc: DW MMC controller at irq 41,32 bit host data width,256 deep fifo
> [    3.229762] dwmmc_rockchip ff520000.mmc: IDMAC supports 32-bit address mode.
> [    3.229799] dwmmc_rockchip ff520000.mmc: Using internal DMA controller.
> [    3.229817] dwmmc_rockchip ff520000.mmc: Version ID is 270a
> [    3.229896] dwmmc_rockchip ff520000.mmc: DW MMC controller at irq 42,32 bit host data width,256 deep fifo
> [    3.231547] mmc_host mmc1: card is non-removable.
> [    3.243883] mmc_host mmc0: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0)
> [    3.244767] mmc_host mmc1: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0)
> [    3.327505] mmc_host mmc1: Bus speed (slot 0) = 150000000Hz (slot req 150000000Hz, actual 150000000HZ div = 0)
> [    3.834347] dwmmc_rockchip ff520000.mmc: Successfully tuned phase to 251
> [    3.834477] mmc1: new HS200 MMC card at address 0001
> [    3.836188] mmcblk1: mmc1:0001 DJNB4R 116 GiB
> [    3.837140] mmcblk1boot0: mmc1:0001 DJNB4R partition 1 4.00 MiB
> [    3.838155] mmcblk1boot1: mmc1:0001 DJNB4R partition 2 4.00 MiB
> [    3.838599] mmcblk1rpmb: mmc1:0001 DJNB4R partition 3 4.00 MiB, chardev (243:0)
> [    3.841290]  mmcblk1: p1
> [    4.876902] EXT4-fs (mmcblk1p1): mounted filesystem with writeback data mode. Opts: (null)
> [    6.614516] EXT4-fs (mmcblk1p1): re-mounted. Opts: commit=600,errors=remount-ro
>
>
> ===
> Mark Millard
> marklmi at yahoo.com
> ( dsl-only.net went
> away in early 2018-Mar)
>