8.0-RELEASE-i386-memstick fixit - No USB devices found!

[Randi Harper]

On Thu, Dec 10, 2009 at 2:20 AM, Ian Smith <smithi at nimnet.asn.au> wrote:
> On Sat, 5 Dec 2009, Randi Harper wrote:
>  > On Sat, Dec 5, 2009 at 4:01 AM, Ian Smith <smithi at nimnet.asn.au> wrote:
>  > > In freebsd-questions Digest, Vol 286, Issue 12, Message 7
>  > > On Fri, 27 Nov 2009 06:51:50 -0800 Randi Harper <randi at freebsd.org> wrote:
>  > >  > On Fri, Nov 27, 2009 at 5:00 AM, Derek (freebsd lists) <482254ac at razorfever.net> wrote:
> [..]
>  > > I made the memstick.img by dd as per the release page on my Thinkpad
>  > > T23.  It only sports USB 1.0 ports, and while I was confident of the dd
>  > > (which took ~25m at ~600kB/s), I didn't really expect a 2002 laptop to
>  > > boot from the image, but on seeing the USB stick show up in its BIOS and
>  > > promoting it in the disk boot order, it did!  Never underestimate IBM ..
> [..]
>  > > So .. booted into sysinstall, fdisk and label ad0s4, leave boot0 as was,
>  > > committed that much after two earlier attempts failed due to the below,
>  > > quit to reboot, checked the labelling, redid the mount points, all ok.
>  > >
>  > > Picked pretty much all distributions from custom install, then of course
>  > > had to select media.  Picked USB - and got about what Derek did, no USB
>  > > disk found.
>  > >
>  > > Very long story short: googled for ages and found a forum thread about
>  > > this very problem, in which someone suggested Options / Rescan Devices
>  > > then trying again.  The OP there said it didn't work for him, but it
>  > > sure did for me!  After knowing that, the install went pretty smoothly,
>  > > modulo not getting fc-10 to install by FTP, but that's another issue..
>  > >
>  > > And just now, prompted by this thread I tried selecting Fixit, to again
>  > > get what Derek did.  And again, Options / Rescan Devices fixed it for
>  > > me.  Maybe it will for Derek and/or maybe provide another clue?  Maybe
>  > > sysinstall could try a device rescan itself in that circumstance?
>  >
>  > This is a known issue. It would be possible to write in a hack to fix
>  > this problem that would be fairly quick to implement, but sysinstall
>  > already has one too many bandaids in place. I'd rather take a little
>  > bit of extra time and fix the underlying problem, especially since
>  > there is this workaround (forcing a device rescan) that seems to work
>  > for users in the meanwhile.
>
> At best it's an 'unknown known' :)  Except for this present thread, my
> 'googling for ages' found nothing in FreeBSD lists about it.  I was so
> close to giving up until I could go somewhere to burn a DVD, by then.
>
> I appreciate your disinclination to extend that message in sysinstall,
> it's been "about to die" for so long it's no longer funny, still it
> would have saved me half a day, and I'm sure I won't be the last person
> to run into this.  I guess I should file a PR with a patch ..

If you want to supply a patch that changes the error message in not
finding a valid USB device, go for it. I'll push it through. One of
the ideas I've been kicking around is consolidating some of the menu
options and having sysinstall look for valid media on all removable
devices (USB/CDROM/floppy), with the option to manually set it still
available in the Options menu. I'm curious to see what other people
think of this first, though. It does "dumb down" the install, but it
still gives advanced users the option of specifying media.

If you want to chuckle, take a look at sysinstall's devices.c and how
it defines a valid USB device (hint: think DD). :)

>
>  > sysinstall was written back in the good 'ol days of pre-devfs and
>  > hasn't been updated much since. When it first runs, it does a device
>  > scan - that is, there's this really ugly data structure of all
>  > possible devices and a description/limit for each. So, just for
>  > example (and I'm not checking the code, so this value is probably
>  > wrong), say there's an entry for 'fxp' that is a type network with a
>  > limit of 16 devices - it's going to poke the system looking for fxp0,
>  > fxp1, ..., fxp15. It's doing this for every single network card, all
>  > possible disk devices, everything. Back in the day when computers were
>  > slower, this process could take a while, so it only happened once
>  > unless the user selected it again.
>
> But now, a rescan on my T23 was quite fast, and it's only a P3 1133MHz.

I wish we could just disregard all the limitations of older hardware
(this would make life so much easier!), but the fact remains that
there are people that still install freebsd on ridiculously old
computers. The smallest group tends to be the most vocal, or so I've
found.

>
>  > Needless to say, this is extremely inefficient (sysinstall code has to
>  > be changed any time a new driver is added, too!) and there's a lot of
>  > better ways to do this. It's very easy to pull a list of network
>  > cards, disks, etc, but the work in moving away from that ugly data
>  > structure is no small job. Right now, much of my time is being taken
>  > up in trying to get gpt support into sysinstall, but getting rid of
>  > that data structure is high in my priority list, especially since
>  > there's a workaround. Old/cheap USB flash sticks seem to be the main
>  > offender, as they are slow to be recognized/probed, and sysinstall has
>  > already finished it's device scan by then.
>
> Point taken, but an unknown workaround is no use to the newbie installer
> (see my prior whinge to Manolis re the X installation menu going away)

I'm unfamiliar with this Manolis person, although I think I poked
through that thread and considered responding but I wanted to research
it a bit more first. I'm getting back from a week-long vacation, and
my sysinstall tagged unread email is... a longer list than I expected.
eek.

>
> Yes, I was using an older 1GB Shintaro stick, the only one on hand, on a
> machine using USB 1.0 only, but I doubt I'll be the only one; sysinstall
> has always striven to work with other than just top-range newer kit.  I
> do know how hard it is for developers to remember what mortals don't
> know, or to imagine why folks might not be running more recent gear :)

I barely pass as a developer, as I'm a sysadmin by trade. Even so,
it's hard to keep in mind all older hardware limitations. Getting rid
of that ugly device scan option will be quite a large patch, but it
will eliminate at least this one problem.

>
>  > > While I'm at it .. selecting 'Holographic Shell', while in that state at
>  > > least, brings up a shell that (perhaps due to stick not being mounted?)
>  > > has no ls command, making navigation difficult :)  pwd works, set works,
>  > > but no ls.  Later (from debug msgs on vty1) I saw that I'd been perhaps
>  > > in /stand and only much later found that find worked and served as ls ..
>  >
>  > This is normal. That's running off of the memory disk. echo * also
>  > works to list files. The memory disk needs to stay fairly small in
>  > size, so it has only a few programs, mostly stuff that is used by
>  > sysinstall.
>
> echo *, now why didn't I think of that :)

Be glad. You've probably never had to deal with remotely upgrading an
older version of linux and the resulting library madness that ended up
with a broken everything. :)

>
> Yeah, I spent a few hours today browsing the src/release stuff.  The
> annotated Makefile alone is a potted history of FreeBSD, and seeing the
> strange mix of floppies through CDs to DVD is interesting, especially
> when the old so constrains the new with perhaps excessive minimalism.
>
>  > > One more thing, while I remember .. seeing the USB stick is here ad0a,
>  > > isn't that the old 'dangerously dedicated mode' now dropped from 8.0?
>  >
>  > Well... yes. But this works. DD mode still works sometimes, but it's
>  > easier to say "not supported" than "only in these circumstances".
>  > FWIW, I don't expect DD mode to be gone forever. There were changes
>  > made outside of sysinstall that aren't compatible with the way that
>  > sysinstall (specifically, libdisk) did things. I think. It's
>  > complicated, not something I'm entirely sure about, and  I
>  > unfortunately can't do anything about other than moving away from
>  > libdisk to libgeom. But this won't fix prior installations. I'm hoping
>  > that those working on geom can come up with a solution for existing DD
>  > installs.
>
> Fair enough, it's a big shift.  I appreciate your detailed explanations.

Sorry I couldn't do better.

>
>  > > And to be a real pest with questions, where in CVS do I find the script
>  > > or whatever makes memstick.img in the first place?
>  >
>  > It's part of the make release process. You really don't want to touch
>  > that. I don't even want to touch that. When I'm testing out
>  > sysinstall, I take an existing memstick and modify it as opposed to
>  > building an entirely new one.
>
> I don't want to touch that, not at least till (if ever) I know exactly
> what's going on there - but I do want to know how such images are made.
>
> Can you share what arguments to make release are used, or what other
> steps are needed to duplicate it?  I'm wondering about a 4GB image say,
> but perhaps this would be a question better posed in -stable?

I'm not sure. I think I saw some magic at some point, but it was a
long time ago, and I can't recall where I saw it - or maybe I forgot
it on purpose. Try -stable, see if you get any good answers. If not,
ping me and I'll see what I can dig up.

-- randi