[REGRESSION] Root zpool mounting broken between 06/30/2013 and 07/21/2013 when PS/2 support compiled into the kernel

[Garrett Cooper]

On Jul 28, 2014, at 6:32 PM, John Baldwin <jhb at freebsd.org> wrote:

> On Monday 28 July 2014 09:57:08 Garrett Cooper wrote:
>> On Mon, Jul 28, 2014 at 9:37 AM, John Baldwin <jhb at freebsd.org> wrote:
>>> On Jul 28, 2014, at 9:07 AM, Garrett Cooper <yaneurabeya at gmail.com> wrote:
>>>> On Jul 22, 2013, at 10:58 AM, Garrett Cooper <yaneurabeya at gmail.com> 
> wrote:
>>>>> On Jul 22, 2013, at 9:08 AM, John Baldwin <jhb at freebsd.org> wrote:
>>>>>> On Monday, July 22, 2013 10:30:32 am Garrett Cooper wrote:
>>>>>>> I have a KERNCONF that previously had PS/2 support compiled into the
>>>>>>> kernel. If I comment out the following lines like so:
>>>>>>> 
>>>>>>> # atkbdc0 controls both the keyboard and the PS/2 mouse
>>>>>>> #device         atkbdc          # AT keyboard controller
>>>>>>> #device         atkbd           # AT keyboard
>>>>>>> 
>>>>>>> then I'm able to mount root again (it was failing with ENOXDEV).
>>>>>>> 
>>>>>>> The working kernel was as follows:
>>>>>>> 
>>>>>>> $ strings /boot/kernel.WORKING/kernel | grep -B 2 -A 2 BAYONETTA
>>>>>>> @(#)FreeBSD 9.1-STABLE #7 r+0304216: Sun Jun 30 15:22:55 PDT 2013
>>>>>>> FreeBSD 9.1-STABLE #7 r+0304216: Sun Jun 30 15:22:55 PDT 2013
>>>>>>> 
>>>>>>> gcooper at bayonetta.local:/usr/obj/scratch/git/github/yaneurabeya-freeb
>>>>>>> sd-stable-9/sys/BAYONETTA>>>>> 
>>>>>>> gcc version 4.2.1 20070831 patched [FreeBSD]
>>>>>>> FreeBSD
>>>>>>> 9.1-STABLE
>>>>>>> BAYONETTA
>>>>>>> $ cd /usr/src; git log 0304216
>>>>>>> commit 03042167f73c213732b44218a24d8e1bbea00f8c
>>>>>>> Merge: 2edcad2 974abfb
>>>>>>> Author: Garrett Cooper <yanegomi at gmail.com>
>>>>>>> Date:   Mon Jun 24 19:00:45 2013 -0700
>>>>>>> 
>>>>>>> Merge remote-tracking branch 'upstream/stable/9' into stable/9
>>>>>>> 
>>>>>>> The working kernel [with atkbdc] was as follows:
>>>>>>> 
>>>>>>> FreeBSD bayonetta.local 9.2-BETA1 FreeBSD 9.2-BETA1 #12 r+c178034: Sun
>>>>>>> Jul 21 20:19:38 PDT 2013>>>> 
>>>>>> root at bayonetta.local:/usr/obj/scratch/git/github/yaneurabeya-freebsd-st
>>>>>> able-9/sys/BAYONETTA  amd64>>>> 
>>>>>>> $ git log c178034
>>>>>>> commit c17803445f4ffb97e1a46a1be5f7ea04692793f0
>>>>>>> Author: avg <avg at FreeBSD.org>
>>>>>>> Date:   Tue Jul 9 08:30:31 2013 +0000
>>>>>>> 
>>>>>>> zfsboottest.sh: remove checks for things that are not strictly
>>>>>>> required
>>>>>>> 
>>>>>>> MFC after:  10 days
>>>>>>> 
>>>>>>> (Yes, I had to backport some things because they are busted on
>>>>>>> stable/9 due to other incomplete/missing MFCs).
>>>>>>> 
>>>>>>> I can test out patches, but I don't have time to bisect the actual
>>>>>>> commit that caused the failure. That being said my intuition says
>>>>>>> it's this>>>> 
>>>>>> commit should be looked at first:
>>>>>>> commit 28f961058b0667841d7e9d8639bfd02ed8689faa
>>>>>>> Author: jhb <jhb at FreeBSD.org>
>>>>>>> Date:   Wed Jul 17 14:04:18 2013 +0000
>>>>>>> 
>>>>>>> MFC 252576:
>>>>>>> Don't perform the acpi_DeviceIsPresent() check for PCI-PCI bridges. 
>>>>>>> If
>>>>>>> we are probing a PCI-PCI bridge it is because we found one by
>>>>>>> enumerating
>>>>>>> the devices on a PCI bus, so the bridge is definitely present.  A few
>>>>>>> BIOSes report incorrect status (_STA) for some bridges that claimed
>>>>>>> they
>>>>>>> were not present when in fact they were.
>>>>>>> 
>>>>>>> While here, move this check earlier for Host-PCI bridges so attach
>>>>>>> fails
>>>>>>> before doing any work that needs to be torn down.
>>>>>>> 
>>>>>>> PR:         kern/91594
>>>>>>> Approved by:        re (marius)
>>>>>> 
>>>>>> I strongly doubt that this is related.  It would be most helpful if you
>>>>>> could obtain a dmesg from the new kernel however (perhaps via a serial
>>>>>> console) to rule it out.  All you would need to see is if the new
>>>>>> kernel sees more "pcib" devices than the old one to see if this change
>>>>>> even has an effect on your system.
>>>>> 
>>>>> Unfortunately the USB keyboard is broken as well at the mount root
>>>>> prompt and the workstation doesn't have a uart on it that I can play
>>>>> with (it's my home box), so I'm dead in the water when it panics at the
>>>>> mount root prompt right now.
>>>>> 
>>>>> I guess I can revert this and a handful of other amd64/ata_cam/zfs
>>>>> commits to see if this goes away, but I won't be getting to that before
>>>>> next Sunday probably as this is my file server and DNS server now.>>> 
>>>>      I ran into the issue going from vanilla 9.2-RELEASE-p10 to
>>>>      9.3-RELEASE as well :(. I’ve filed this bug to track the issue:
>>>>      https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192183 . I’ll
>>>>      see if GENERIC can boot my system sometime this week (the KERNCONF
>>>>      has been working for several releases, but it could be an issue
>>>>      with that that’s being overlooked by accident).>> 
>>>> Thanks!
>> 
>> ...
>> 
>>> Also, you would need to get verbose dmesg's of old and new kernels as a
>>> first step in narrowing it down.
>> I can't do trivial debugging because my USB keyboard doesn't work at
>> the mountroot prompt (
>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=133989 ), I don't
>> have a PS/2 keyboard, and the system only has VGA access :(. I'll try
>> working on disabling the PS/2 controller in the BIOS and a few other
>> things to force the system to stop ignoring the USB keyboard to get
>> scrollback, because that appeared to work for some folks with this
>> issue according to the ukbd bug I referenced.
> 
> Do you have a serial port so you could use a serial console (or is this a 
> laptop)?

I wish my motherboard had an RS232 port, but unfortunately it doesn’t; I don’t have 2 USB RS-232 converters either (and I’m not sure a USB serial adapter would work for boot2sio, would it?) :/.

If the legacy USB keyboard route doesn’t work out, I have a USB to PS/2 converter coming in the mail.

It would be nice if bug 133989 was fixed. I’ll talk to hps@ about it…

Thanks!
-Garrett