[Bug 280140] mpr max_chains causing boot failure when zfs module is loaded during boot

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=280140

            Bug ID: 280140
           Summary: mpr max_chains causing boot failure when zfs module is
                    loaded during boot
           Product: Base System
           Version: 14.1-RELEASE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: bsd.m0osk@gmail.com

Created attachment 251883
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=251883&action=edit
screenshot showing mpr out of chain frames immediately after zfs is loaded

Server Hardware
Model: PowerEdge R760xd2 
RAM: 256GB
Disk Controller: Broadcom / LSI HBA355i Adapter


Symptom(s):
during boot, mpr runs out of chain frames as soon as zfs module is loaded in
the kernel and thereby fails to boot into OS


Details:
If I boot from the installation disk, and during boot set
hw.mpr.max_chains=32768 and dev.mpr.0.max_chains=32768 then I can boot to
installer and install the OS (either ZFS or UFS)

However, once I have setup a zfs-on-root system, the server fails to boot on
subsequent attempts (from the installed OS) irrespective of any number/value
set for max_chains, it always complains about running out of chain frames. This
happens immediately after loading the ZFS module (feature support 5000).

I have tried all possible values for max_chains between 8192 till 1048576, none
of them worked, and same error message about running out of chain frames. 

I have tried to boot by disabling zfs on the installed system, that fails and
goes to mountroot (obviously!). Since I am not aware of any other way to load
the zfs module between mountroot and *right after* mpr has settled down, I do
not know what other test can be done to confirm or get more details and
possibly fix this.

What has worked so far:
I can boot into the installer and install a root-on-UFS system. As long as I am
not loading the zfs module during boot (either manually or via
/boot/loader.conf), the system boots without any problem. I can have zfs
pools/datasets by having zfs and zfsd running from rc.conf (it loads the module
after the system has settled down)

So, in a way I have a relatively functional/semi-stable system that I can
continue to work with. But I would very much like to have the root on zfs.

Curious facts:
I have an almost-identical server, difference being the other one has 512GB ram
and does not have some of the ethernet cards that are in this server, and that
other server is running with FreeBSD 14.1 with root-on-zfs. The other server
being on production I cannot run any tests on it now. So, for comparison, I
disabled the additional network cards on the current partially working system,
and tried to boot into it. Same error -- running out of chain frames.

The option to put root-on-UFS works anyway.

-- 
You are receiving this mail because:
You are the assignee for the bug.