ZFS boot problems with memory > 1MB

[Brandon Gooch]

On Tue, Feb 23, 2010 at 10:24 AM, John Baldwin <jhb at freebsd.org> wrote:
> On Tuesday 23 February 2010 10:28:49 am Brandon Gooch wrote:
>> On Tue, Feb 23, 2010 at 7:29 AM, Andriy Gapon <avg at icyb.net.ua> wrote:
>> > on 23/02/2010 13:18 Renato Botelho said the following:
>> >> On Mon, Feb 22, 2010 at 7:35 PM, Chris Hedley
>> >> <freebsd-current at chrishedley.com> wrote:
>> > [snip]
>> >>> Do you have USB legacy support enabled in your BIOS?  I'm not sure if
>> >>> there's an option for the loader to use USB devices natively, but the BIOS's
>> >>> legacy option where it provides AT/PS2 emulation is probably the easiest way
>> >>> to get the keyboard working.
>> >>
>> >> Yes, I do, but it seems to be a regression on FreeBSD itself, I had this problem
>> >> in the past and I checked the same things i need to check in the past again and
>> >> everything is fine.
>> >
>> > A more precise way to state that would be "a regression in FreeBSD boot/loader".
>> > I think that you are referring to the issue that was fixed by r189017.
>> > It might be worthwhile investigating what was done in that revision and what
>> > happened in sys/boot code since then.
>> >
>> > One possibility is that your BIOS uses memory above 1MB for USB emulation, but
>> > doesn't mark that memory as used in system memory map.  In that case that memory
>> > could be overwritten by the loader.  If that's true then the blame is on the BIOS.
>> >  Alternatively, our code might be parsing the system memory map incorrectly.
>> > But I am just making wild guesses here.
>> >
>>
>> I don't know if it is at all related, but this commit has caused
>> problems for me booting at least one of my machines:
>>
>> http://svn.freebsd.org/viewvc/base/head/sys/boot/i386/zfsboot/zfsboot.c?r1=199714&r2=200309
>>
>> Commit message:
>>
>> Revision 200309 - (view) (annotate) - [select for diffs]
>> Modified Wed Dec 9 20:36:56 2009 UTC (2 months, 2 weeks ago) by jhb
>> File length: 24893 byte(s)
>> Diff to previous 199714
>> - Port bios_getmem() from libi386 to {gpt,}zfsboot() and use it to
>>   safely allocate a heap region above 1MB.  This enables {gpt,}zfsboot()
>>   to allocate much larger buffers than before.
>> - Use a larger buffer (1MB instead of 128K) for temporary ZFS buffers.  This
>>   allows more reliable reading of compressed files in a raidz/raidz2 pool.
>>
>> Submitted by: Matt Reimer  mattjreimer of gmail
>> MFC after:    1 week
>
> Starting a new thread, which problems are you seeing with this change?  ZFS is
> a good bit more memory hungry than UFS, so it really needs to use high memory
> for its heap.  Also, I wonder if you still have problems if you use the older
> zfsboot with the newer zfsloader?  Finally, you need to use disklabel -B or
> some such to update the zfsboot bits for this change to take effect.
>
> --
> John Baldwin
>

I filed a PR so it wouldn't fall through the cracks:

http://www.freebsd.org/cgi/query-pr.cgi?pr=144234

I guess I tried a combination of various revisions of bootstrap code
and loaders when I first encountered the issue. It was when I wrote a
recent gptzfsboot to the geom that I saw the symptoms:

error 1 lba 48
error 1 lba 1
No ZFS pools located, can't boot

I just wound up using sys/boot/i386/zfsboot/zfsboot.c revision 199714
to build a working gptzfsboot on another system and wrote that to the
disk to get the machine operational.

-Brandon