Re: Questions about probing WLUNs for UFS power management
Date: Thu, 02 Oct 2025 08:13:56 UTC
Hi Warner,
I’m not fully sure this is the best path yet, but I looked more into option (1).
Below are the questions I had and the answers I found.
Q. Are the semantics of START STOP UNIT (SSU) in the UFS spec the same
as in SBC-5?
A. Yes. SBC-5 defines SSU in a flexible way, and the UFS spec defines
its SSU usage to be compliant with SBC-5.
Q. Where is SSU used in (linux) UFS driver?
A. Power management. During suspend the driver sends 'UFS-Sleep (POWER
CONDITION = 0x2)' or 'UFS-PowerDown (0x3)', and during resume the
driver sends 'Active (0x1)'.
Q. Can we reuse an existing periph driver?
A. No. Existing periph drivers aren’t suitable for WLUNs. The pass
thru device also doesn't implement suspend/resume.
Q. How do we register handlers for suspend/resume in FreeBSD?
A. A periph driver can register handlers at init time with
EVENTHANDLER_REGISTER(power_suspend, ...) and
EVENTHANDLER_REGISTER(power_resume, ...).
Now that those questions have been clarified, I fully understand your comments.
At this point, it seems we need to create a periph driver like
`scsi_wlun.c` for attaching WLUNs, and implement suspend/resume
function within it.
In suspend/resume, we should issue the SSU command.
Planned tasks:
1) Define a `PIM_WLUN` flag so only drivers that want WLUNs discover them.
2) Use `REPORT LUNS` with `SELECT REPORT` = 0x01 to obtain the WLUN list first.
3) Add a `CAM_GET_WLUN()` macro to parse the WLUN list.
4) When `CAM_GET_WLUN()` finds a match, automatically attach the WLUN periph.
5) In `scsi_wlun.c` 'wluninit()', register power handlers
(EVENTHANDLER_REGISTER(power_suspend, wlunsuspend, ...),
EVENTHANDLER_REGISTER(power_resume, wlunresume, ...)).
6) In `wlunsuspend() / wlunresume()`, issue SSU commands to the UFS Device WLUN.
7) The order between LUN periph (da) and WLUN periph must be guaranteed.
(Ensure that the WLUN SSU is issued last during suspension and
first during resumption.)
It's Korean Thanksgiving day until October 10. I'll take a short break
and then start prototyping this right after.
If I've misunderstood anything, please let me know.
Jaeyoon
On Wed, Oct 1, 2025 at 12:11 PM Jaeyoon Choi <jaeyoon@freebsd.org> wrote:
>
> > It's easy enough to add the enumeration for (1) to scsi_xpt.c It's just a few more states in the probe state machine (which is a bit complex, but not terribly so). If we treat them as separate LUNs, then a lot of the infrastructure would just work. The key, though, is how do we keep scsi_da, etc from attaching to these new nodes. And would we have generic scsi_foo.c drivers for them (so the devices would get their own periph drivers) or whether there's a higher-level device that would know it could use these devices if it finds them and it wouldn't have a specific specialized kernel drive for them, but instead use a generalized pass thru device.
>
> That's correct. Probing the WLUN isn't difficult. And being able to
> use the SCSI infrastructure is a huge advantage.
> However, as you mentioned, having the peripheral driver (scsi_da?)
> attached to the WLUN node seems like it could be a problem.
>
> >
> > I guess a lot of that would depend where things like power management is happening. If it's in the UFS sim, that would suggest a different design than if this was some super generic thing that would respond to generic events that maybe the SIMs generate or maybe somebody else.
>
> Refer to Section 7.4.2 “Power Management Command: START STOP UNIT” in
> the UFS 2.1 specification. The START STOP UNIT command is used to
> change the power mode.
> The SBC-5 spec also defines the SSU's POWER CONDITION field. (SBC-5
> 5.31 “START STOP UNIT command”)
> The specs seem similar, but I need to study further to understand
> whether UFS's power management approach is a super generic thing.
>
> I'll come back after reading more of the spec and code.
>
> Thanks,
> Jaeyoon
>
> On Mon, Sep 29, 2025 at 6:48 AM Warner Losh <imp@bsdimp.com> wrote:
> >
> > Please excuse the tardy response
> >
> > On Tue, Sep 23, 2025 at 8:32 PM Jaeyoon Choi <jaeyoon@freebsd.org> wrote:
> >>
> >> Hello freebsd-scsi,
> >>
> >> I’m trying to implement Well-known LUNs (WLUNs) for the UFS (Universal
> >> Flash Storage, ufshci(4)) driver, but the current SCSI XPT doesn't probe
> >> WLUNs, so I have a few questions.
> >>
> >> First, Well-known LUNs are defined in SCSI SAM-5. In UFS, 'REPORT LUNS',
> >> 'UFS Device', 'RPMB', and 'BOOT' LUNs are treated as WLUNs.
> >> (SAM-5 4.7.5.1 "Well-known logical unit addressing",
> >> UFS 2.1 spec 10.8.5 "10.8.5 Well Known Logical Unit Defined in UFS")
> >>
> >> Because a UFS device contains multiple LUNs, power management requires
> >> sending a START STOP UNIT(SSU) command to the 'UFS Device' WLUN. (If SSU
> >> is sent to a non-WLUN (normal) LUN, the device power mode is not changed
> >> and only that LUN becomes disabled.)
> >> (UFS 2.1 spec 7.4.2 "Power Management Command: START STOP UNIT")
> >>
> >> In FreeBSD, the SCSI XPT logic that parses LUNs checks the 'Simple logical
> >> unit addressing format' (SAM-5 Table 26) and the 8-byte 'Extended logical
> >> unit addressing format' (SAM-5 Table 35), but it does not check the
> >> 'Well-known logical unit extended addressing format' (SAM-5 Table 37).
> >> In the scsi_xpt.c code below, CAM_GET_LUN reads Extended LUNs and
> >> CAM_GET_SIMPLE_LUN reads Simple LUNs:
> >>
> >> ```
> >> sys/cam/scsi/scsi_xpt.c line:2114
> >> static void
> >> scsi_scan_bus(struct cam_periph *periph, union ccb *request_ccb)
> >> ...
> >> while (scan_info->lunindex[target_id] < nluns) {
> >> if (scan_info->cpi->hba_misc & PIM_EXTLUNS) {
> >> CAM_GET_LUN(target->luns,
> >> scan_info->lunindex[target_id],
> >> lun_id);
> >> break;
> >> }
> >>
> >> if (CAM_CAN_GET_SIMPLE_LUN(target->luns,
> >> scan_info->lunindex[target_id])) {
> >> CAM_GET_SIMPLE_LUN(target->luns,
> >> scan_info->lunindex[target_id],
> >> lun_id);
> >> break;
> >> }
> >>
> >> scan_info->lunindex[target_id]++;
> >> }
> >> ```
> >>
> >> Also, we can obtain the number of WLUNs via the REPORT LUNS SCSI command.
> >> To retrieve WLUNs, the SELECT REPORT field must be set to 0x01, but
> >> SCSI XPT currently uses 0x00 to obtain only normal LUNs.
> >> (UFS 2.1 spec 11.3.12.2 "Report LUNS Command Select Report Field Values")
> >>
> >> ```
> >> sys/cam/scsi/scsi_xpt.c line:827
> >> static void
> >> probestart(struct cam_periph *periph, union ccb *start_ccb)
> >> ...
> >> case PROBE_REPORT_LUNS:
> >> ...
> >> scsi_report_luns(csio, 5, probedone, MSG_SIMPLE_Q_TAG,
> >> RPL_REPORT_DEFAULT, rp, periph->path->target->rpl_size,
> >> SSD_FULL_SIZE, 60000);
> >> ```
> >> ```
> >> sys/cam/scsi/scsi_all.h line:3020
> >> struct scsi_report_luns
> >> {
> >> uint8_t opcode;
> >> uint8_t reserved1;
> >> #define RPL_REPORT_DEFAULT 0x00
> >> #define RPL_REPORT_WELLKNOWN 0x01
> >> ```
> >>
> >> Looking at the Linux code, Linux also does not check WLUNs at the SCSI
> >> layer. Therefore, the Linux UFS driver explicitly registers the WLUNs
> >> during initialization by calling __scsi_add_device()
> >>
> >> ```
> >> drivers/ufs/core/ufshcd.c line:8137
> >> /**
> >> * ufshcd_scsi_add_wlus - Adds required W-LUs
> >> * @hba: per-adapter instance
> >> *
> >> * UFS device specification requires the UFS devices to support 4 well known
> >> * logical units:
> >> * "REPORT_LUNS" (address: 01h)
> >> * "UFS Device" (address: 50h)
> >> * "RPMB" (address: 44h)
> >> * "BOOT" (address: 30h)
> >> * UFS device's power management needs to be controlled by "POWER CONDITION"
> >> * field of SSU (START STOP UNIT) command. But this "power condition" field
> >> * will take effect only when its sent to "UFS device" well known logical unit
> >> * hence we require the scsi_device instance to represent this logical unit in
> >> * order for the UFS host driver to send the SSU command for power management.
> >> *
> >> * We also require the scsi_device instance for "RPMB" (Replay Protected Memory
> >> * Block) LU so user space process can control this LU. User space may also
> >> * want to have access to BOOT LU.
> >> *
> >> * This function adds scsi device instances for each of all well known LUs
> >> * (except "REPORT LUNS" LU).
> >> *
> >> * Return: zero on success (all required W-LUs are added successfully),
> >> * non-zero error value on failure (if failed to add any of the required W-LU).
> >> */
> >> static int ufshcd_scsi_add_wlus(struct ufs_hba *hba)
> >> {
> >> int ret = 0;
> >> struct scsi_device *sdev_boot, *sdev_rpmb;
> >>
> >> hba->ufs_device_wlun = __scsi_add_device(hba->host, 0, 0,
> >> ufshcd_upiu_wlun_to_scsi_wlun(UFS_UPIU_UFS_DEVICE_WLUN), NULL);
> >> if (IS_ERR(hba->ufs_device_wlun)) {
> >> ret = PTR_ERR(hba->ufs_device_wlun);
> >> hba->ufs_device_wlun = NULL;
> >> goto out;
> >> }
> >> scsi_device_put(hba->ufs_device_wlun);
> >>
> >> sdev_rpmb = __scsi_add_device(hba->host, 0, 0,
> >> ufshcd_upiu_wlun_to_scsi_wlun(UFS_UPIU_RPMB_WLUN), NULL);
> >> if (IS_ERR(sdev_rpmb)) {
> >> ret = PTR_ERR(sdev_rpmb);
> >> goto remove_ufs_device_wlun;
> >> }
> >> ufshcd_blk_pm_runtime_init(sdev_rpmb);
> >> scsi_device_put(sdev_rpmb);
> >>
> >> sdev_boot = __scsi_add_device(hba->host, 0, 0,
> >> ufshcd_upiu_wlun_to_scsi_wlun(UFS_UPIU_BOOT_WLUN), NULL);
> >> if (IS_ERR(sdev_boot)) {
> >> dev_err(hba->dev, "%s: BOOT WLUN not found\n", __func__);
> >> } else {
> >> ufshcd_blk_pm_runtime_init(sdev_boot);
> >> scsi_device_put(sdev_boot);
> >> }
> >> goto out;
> >>
> >> remove_ufs_device_wlun:
> >> scsi_remove_device(hba->ufs_device_wlun);
> >> out:
> >> return ret;
> >> }
> >> ```
> >>
> >> ---------------------------------------------------------------------------
> >>
> >> My question is: which approach would be more appropriate for using WLUN
> >> devices in UFS driver on FreeBSD?
> >>
> >> (1) Add support in SCSI XPT to probe WLUNs and allocate devices for them.
> >> (2) Follow Linux: have the UFS driver explicitly probe/register WLUN
> >> devices itself.
> >>
> >> If (2) is preferred, what is the recommended way for the UFSHCI driver to
> >> probe/register WLUNs in FreeBSD? (Is there an equivalent to Linux’s
> >> __scsi_add_device()?)
> >
> >
> > [[ I didn't trim context here, since it lays out the problem nicely). ]]
> >
> > So I can make a case for either (1) or (2).
> >
> > It's easy enough to add the enumeration for (1) to scsi_xpt.c It's just a few more states in the probe state machine (which is a bit complex, but not terribly so). If we treat them as separate LUNs, then a lot of the infrastructure would just work. The key, though, is how do we keep scsi_da, etc from attaching to these new nodes. And would we have generic scsi_foo.c drivers for them (so the devices would get their own periph drivers) or whether there's a higher-level device that would know it could use these devices if it finds them and it wouldn't have a specific specialized kernel drive for them, but instead use a generalized pass thru device.
> >
> > I guess a lot of that would depend where things like power management is happening. If it's in the UFS sim, that would suggest a different design than if this was some super generic thing that would respond to generic events that maybe the SIMs generate or maybe somebody else.
> >
> > If we wanted to go with (1) heavily, we'd need new, generic periph drivers along with some kind of way to get them to respond to, or report out events in the system or with the drive. With only a couple of examples, this might be difficult to abstract properly.
> >
> > If we wanted to go with (2), then we'd likely need some extensions in xpt around this so that the sim could discover, manage and use the WLUN devices.
> >
> > Though the more I think about it, maybe it doesn't matter: we want to do the WLUN probing regardless. It's who manages the cam_ed structures after that. Where does the periph driver live, etc.
> >
> > I think I need to go read the UFS specs in more detail than I have time for this afternoon...
> >
> > Warner
> >
> > Warner