8.0-RELEASE-i386-memstick fixit - No USB devices found!

[Ian Smith]

On Sat, 5 Dec 2009, Randi Harper wrote:
 > On Sat, Dec 5, 2009 at 4:01 AM, Ian Smith <smithi at nimnet.asn.au> wrote:
 > > In freebsd-questions Digest, Vol 286, Issue 12, Message 7
 > > On Fri, 27 Nov 2009 06:51:50 -0800 Randi Harper <randi at freebsd.org> wrote:
 > >  > On Fri, Nov 27, 2009 at 5:00 AM, Derek (freebsd lists) <482254ac at razorfever.net> wrote:
[..]
 > > I made the memstick.img by dd as per the release page on my Thinkpad
 > > T23.  It only sports USB 1.0 ports, and while I was confident of the dd
 > > (which took ~25m at ~600kB/s), I didn't really expect a 2002 laptop to
 > > boot from the image, but on seeing the USB stick show up in its BIOS and
 > > promoting it in the disk boot order, it did!  Never underestimate IBM ..
[..]
 > > So .. booted into sysinstall, fdisk and label ad0s4, leave boot0 as was,
 > > committed that much after two earlier attempts failed due to the below,
 > > quit to reboot, checked the labelling, redid the mount points, all ok.
 > >
 > > Picked pretty much all distributions from custom install, then of course
 > > had to select media.  Picked USB - and got about what Derek did, no USB
 > > disk found.
 > >
 > > Very long story short: googled for ages and found a forum thread about
 > > this very problem, in which someone suggested Options / Rescan Devices
 > > then trying again.  The OP there said it didn't work for him, but it
 > > sure did for me!  After knowing that, the install went pretty smoothly,
 > > modulo not getting fc-10 to install by FTP, but that's another issue..
 > >
 > > And just now, prompted by this thread I tried selecting Fixit, to again
 > > get what Derek did.  And again, Options / Rescan Devices fixed it for
 > > me.  Maybe it will for Derek and/or maybe provide another clue?  Maybe
 > > sysinstall could try a device rescan itself in that circumstance?
 > 
 > This is a known issue. It would be possible to write in a hack to fix
 > this problem that would be fairly quick to implement, but sysinstall
 > already has one too many bandaids in place. I'd rather take a little
 > bit of extra time and fix the underlying problem, especially since
 > there is this workaround (forcing a device rescan) that seems to work
 > for users in the meanwhile.

At best it's an 'unknown known' :)  Except for this present thread, my 
'googling for ages' found nothing in FreeBSD lists about it.  I was so 
close to giving up until I could go somewhere to burn a DVD, by then.

I appreciate your disinclination to extend that message in sysinstall, 
it's been "about to die" for so long it's no longer funny, still it 
would have saved me half a day, and I'm sure I won't be the last person 
to run into this.  I guess I should file a PR with a patch ..

 > sysinstall was written back in the good 'ol days of pre-devfs and
 > hasn't been updated much since. When it first runs, it does a device
 > scan - that is, there's this really ugly data structure of all
 > possible devices and a description/limit for each. So, just for
 > example (and I'm not checking the code, so this value is probably
 > wrong), say there's an entry for 'fxp' that is a type network with a
 > limit of 16 devices - it's going to poke the system looking for fxp0,
 > fxp1, ..., fxp15. It's doing this for every single network card, all
 > possible disk devices, everything. Back in the day when computers were
 > slower, this process could take a while, so it only happened once
 > unless the user selected it again.

But now, a rescan on my T23 was quite fast, and it's only a P3 1133MHz.

 > Needless to say, this is extremely inefficient (sysinstall code has to
 > be changed any time a new driver is added, too!) and there's a lot of
 > better ways to do this. It's very easy to pull a list of network
 > cards, disks, etc, but the work in moving away from that ugly data
 > structure is no small job. Right now, much of my time is being taken
 > up in trying to get gpt support into sysinstall, but getting rid of
 > that data structure is high in my priority list, especially since
 > there's a workaround. Old/cheap USB flash sticks seem to be the main
 > offender, as they are slow to be recognized/probed, and sysinstall has
 > already finished it's device scan by then.

Point taken, but an unknown workaround is no use to the newbie installer 
(see my prior whinge to Manolis re the X installation menu going away)

Yes, I was using an older 1GB Shintaro stick, the only one on hand, on a 
machine using USB 1.0 only, but I doubt I'll be the only one; sysinstall 
has always striven to work with other than just top-range newer kit.  I 
do know how hard it is for developers to remember what mortals don't 
know, or to imagine why folks might not be running more recent gear :)

 > > While I'm at it .. selecting 'Holographic Shell', while in that state at
 > > least, brings up a shell that (perhaps due to stick not being mounted?)
 > > has no ls command, making navigation difficult :)  pwd works, set works,
 > > but no ls.  Later (from debug msgs on vty1) I saw that I'd been perhaps
 > > in /stand and only much later found that find worked and served as ls ..
 > 
 > This is normal. That's running off of the memory disk. echo * also
 > works to list files. The memory disk needs to stay fairly small in
 > size, so it has only a few programs, mostly stuff that is used by
 > sysinstall.

echo *, now why didn't I think of that :)

Yeah, I spent a few hours today browsing the src/release stuff.  The 
annotated Makefile alone is a potted history of FreeBSD, and seeing the 
strange mix of floppies through CDs to DVD is interesting, especially 
when the old so constrains the new with perhaps excessive minimalism.

 > > One more thing, while I remember .. seeing the USB stick is here ad0a,
 > > isn't that the old 'dangerously dedicated mode' now dropped from 8.0?
 > 
 > Well... yes. But this works. DD mode still works sometimes, but it's
 > easier to say "not supported" than "only in these circumstances".
 > FWIW, I don't expect DD mode to be gone forever. There were changes
 > made outside of sysinstall that aren't compatible with the way that
 > sysinstall (specifically, libdisk) did things. I think. It's
 > complicated, not something I'm entirely sure about, and  I
 > unfortunately can't do anything about other than moving away from
 > libdisk to libgeom. But this won't fix prior installations. I'm hoping
 > that those working on geom can come up with a solution for existing DD
 > installs.

Fair enough, it's a big shift.  I appreciate your detailed explanations.

 > > And to be a real pest with questions, where in CVS do I find the script
 > > or whatever makes memstick.img in the first place?
 > 
 > It's part of the make release process. You really don't want to touch
 > that. I don't even want to touch that. When I'm testing out
 > sysinstall, I take an existing memstick and modify it as opposed to
 > building an entirely new one.

I don't want to touch that, not at least till (if ever) I know exactly 
what's going on there - but I do want to know how such images are made.

Can you share what arguments to make release are used, or what other 
steps are needed to duplicate it?  I'm wondering about a 4GB image say,
but perhaps this would be a question better posed in -stable?

cheers, Ian