USB drives still don't work correctly

[Bernd Walter]

On Thu, Sep 23, 2010 at 01:15:15PM -0400, Donald Allen wrote:
> On Thu, Sep 23, 2010 at 11:59 AM, Bernd Walter <ticso at cicely7.cicely.de> wrote:
> > On Thu, Sep 23, 2010 at 11:07:59AM -0400, Donald Allen wrote:
> >> On Thu, Sep 23, 2010 at 2:34 AM, Hans Petter Selasky <hselasky at c2i.net> wrote:
> >> > On Wednesday 22 September 2010 18:03:11 Donald Allen wrote:
> >> >> I have tried periodically to use FreeBSD -- a couple of the 7.x
> >> >> releases, 8.0 and now 8.1. I do my backups on Seagate SATA drives in
> >> >> USB shoeboxes with ext2 filesystems. These drives work fine with Linux
> >> >> (Slackware) and OpenBSD. But with FreeBSD, absolutely no luck. The 7.x
> >> >> releases would freeze or crash. The much-needed reimplementation of
> >> >> the USB layer in 8.x gave me new hope, but I'm still experiencing
> >> >> problems.
> >> >>
> >> >> I have 8.1 RELEASE installed on a Thinkpad G41, an old system I use
> >> >> for experimenting. When I plug in one of the USB drives directly to
> >> >> the machine, I get  the following in /var/log/messages:
> >> >>
> >> >> Sep 22 09:53:10 elektra kernel: ugen3.2: <Sunplus Technology Inc.> at
> >> >> usbus3 Sep 22 09:53:10 elektra kernel: umass0: <Bulk Only Interface> on
> >> >> usbus3 Sep 22 09:53:10 elektra kernel: umass0:  SCSI over Bulk-Only;
> >> >> quirks = 0x4000 Sep 22 09:53:10 elektra root: Unknown USB device: vendor
> >> >> 0x04fc
> >> >> product 0x0c15 bus uhub3
> >> >> Sep 22 09:53:12 elektra kernel: umass0:0:0:-1: Attached to scbus0
> >> >> Sep 22 09:53:37 elektra kernel: (da0:umass-sim0:0:0:0): AutoSense failed
> >> >> Sep 22 09:53:37 elektra kernel: (da0:umass-sim0:0:0:0): got CAM status 0x10
> >> >> Sep 22 09:53:37 elektra kernel: (da0:umass-sim0:0:0:0): fatal error,
> >> >> failed to attach to device
> >> >> Sep 22 09:53:37 elektra kernel: (da0:umass-sim0:0:0:0): lost device
> >> >> Sep 22 09:53:47 elektra kernel: (da0:umass-sim0:0:0:0): AutoSense failed
> >> >> Sep 22 09:53:47 elektra kernel: (da0:umass-sim0:0:0:0): removing device
> >> >> entry
> >> >>
> >> >> After the above, if I remove the USB connector and plug it back in, X
> >> >> freezes (the cursor moves with the mouse, but no response to clicks,
> >> >> or to keyboard gestures) until I remove the connector.
> >> >>
> >> >> Interestingly, if I plug the drive in prior to booting the system, the
> >> >> system recognizes it properly and I can mount it and display its root
> >> >> directory. So there is a workaround. But after all this time that I've
> >> >> been trying to use FreeBSD and all the effort that's gone into getting
> >> >> the USB layer right, it's discouraging to still be running into issues
> >> >> like this. Hopefully, one of you will have a bright (configuration?)
> >> >> idea that will allow me to use the USB drives as I do on other
> >> >> systems, without the need for the reboot workaround.
> >> >>
> >> >> Thanks --
> >> >>
> >> >
> >> > Hi,
> >> >
> >> > Maybe your drive needs an USB quirk to work properly. See:
> >> >
> >> > usbconfig -h
> >> >
> >> > Look for "quirk".
> >>
> >> There are 48 supported quirks. Do you have suggestions as to which I might try?
> >>
> >> I would add that I'm not sure if you are suggesting this as a solution
> >> to my problem, or as a step toward improving the USB driver. If the
> >> latter, I'm happy to help. If the former, I'd argue that's not
> >> sufficient. These drives (I have three identical ones) work
> >> flawlessly, with no user intervention, with Linux (Slackware), and
> >> OpenBSD. FreeBSD ought to do the same. And if a "quirk" is needed with
> >> FreeBSD, it seems odd that the drives work correctly and without added
> >> quirks with FreeBSD when plugged in at boot-time (as I said in the
> >> message to which you replied). but not when hot-plugged. This suggests
> >> to me that there's a bug, or, at the very least, a missing feature.
> >
> > Or FreeBSD tries to use a feature, which is broken with the drive,
> > which often happens, since FreeBSD tries hard to be fast and correct.
> > For example broken cache flushes won't hurt if your OS don't bother
> > to even try a flush, broken tagged command queuing or pipelining won't
> > hurt if your OS only does single transactions.
> > Yes: FreeBSD is unusual in that it expects a device to work correctly
> > and that announced features can be used.
> > You also shouldn't ignore that fact that other OS are also use quirk
> > tables to work aroung broken devices
> 
> I'm not ignoring that at all. I've looked at the Linux driver and am
> aware of usb/core/quirks.c. I'm simply stating the undeniable fact
> that the other OSes don't require manual intervention in order to use
> these drives hot-plugged. If they are using heuristics behind the
> scenes to apply the right quirks, that's fine with me.

In my opion FreeBSD has found more HDD firmware bugs than any other OS.
Just think about FreeBSD3 when CAM was introduced and a large set
of SCSI drives failed because they never expected tagged command queuing
to be used that agressive.
I'm aware of at least three major (at that time) vendors which had
devices with problems.
Sometimes it is simple to avoid bugs.
Many USB devices fail to comply with specifications if asked anything
before being addressed - this is a clear violation, but just changing
the order helped without harm.
It the tagged command queuing it is not that simple - an OS not using it
agressive won't notice, but does it mean we can't every use that feature
on all those device which are perfectly Ok?
Whitelisting is also not an option because why should we punish the
good vendors?

>  - of course Linux is more often
> > used with crappy hardware than FreeBSD,
> 
> We don't live in an ideal world and not every device implements the
> specs correctly. Apparently there are enough devices like mine out
> there that Linux and OpenBSD have found a way to make them work. It's
> sometimes not a winning strategy to insist on absolute adherence to a
> protocol -- Opera tried that with their browser and look at their
> share of the desktop market. It can be a bit like having "I had the
> right of way" on your tombstone.

Yes - that point has something in common.
People expect crappy webpages to work and people expect crappy hardware
to work.
But FreeBSD is not an OS which trades working with crappy hardware at
the cost of not using important features with working devcies.
Windows for example handles USB disks completely different than SATA
just becasuse it assumes people might disconnect it at any time.
They only do a cache flush in the device clicked for being disconnected
and if the device hangs on that command - well - the user is about to
disconnect it anyway.

> so they likely have bigger
> > quirk tables - with OpenBSD - well either it doesn't use the broken
> > feature at all or they are already stumbled over the problem by luck.
> > I've never seen a Sunplus USB mass storage device so far and I've seen
> > a lot.
> > The FreeBSD driver likely isn't at fault, since it works with so many
> > non-broken devices.
> > For me it looks like the device is just hanging on a command, which
> > would be clearly against specification.
> > I've seen so many creative firmware bugs in USB devices that I don't
> > easily trust any of them.
> > Fortunately most vendors have learned their lessons, so that more
> > recent devices are less error prune.
> 
> Whether my hardware is "crappy" or not (and you may well be right
> about that -- these shoeboxes are a few years old), there's still the
> issue that the drives work correctly with FreeBSD when present at
> boot-time, but not when hot-plugged. I would think the handshake with
> the drive would be the same in either case, so either I'm wrong or
> there's a bug.

Yes - the handshake should be the same, but the drive state is not.
So it is just when hot-plugging the device and not when booting connected?
Probably it just hangs if asked before the media has spun up, which is
likely the case when it had power long time before.
Or has the device an external power source and is already rotating before
plugged in?

> If you think a command is hanging, can you suggest a quirk or subset
> of the 48 to try that might cure the problems in the hotplug case?

Unfortunately not.
Maybe you have a chance to find out via cam debug which command it
has problems with, but I'm not familar with it's use, since I never
needed it myself.

-- 
B.Walter <bernd at bwct.de> http://www.bwct.de
Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.