[Bug 194455] New: [PATCH] Preserve lowmem for devices on large memory machines

bugzilla-noreply at freebsd.org bugzilla-noreply at freebsd.org
Sat Oct 18 20:45:20 UTC 2014


https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194455

            Bug ID: 194455
           Summary: [PATCH] Preserve lowmem for devices on large memory
                    machines
           Product: Base System
           Version: 11.0-CURRENT
          Hardware: Any
                OS: Any
            Status: Needs Triage
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: freebsd-bugs at FreeBSD.org
          Reporter: conrad.meyer at isilon.com

Created attachment 148440
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=148440&action=edit
( applies with -p1 on CURRENT ~r273263 )

Log:
==================================8<==================================

Add an intermediate VM_FREELIST "DMA32" between DEFAULT and ISADMA on
AMD64. Physical allocations try freelists ascending, so the effect is to
prefer DEFAULT, then DMA32, then ISADMA.

This leaves low memory available for things that need it (DMA) on
large-memory systems which statically allocate a lot (e.g. bufs).

Sponsored by:   EMC / Isilon storage division

==================================8<==================================

Historical note/context:

So, we have 96-256 GB systems, and scale nbufs up with memory. On these systems
we tune for much more than 4GB in bufs. It turns out the 'buf' static array is
allocated from physmem fairly early (along with some other things that scale
with memory), and without this change, devices starved for 32-bit DMA space.

Internally, we had just bumped the ISADMA region from 16MB (24-bit) to 4GB. I
think the attached patch with a seperate 32-bit DMA region is slightly cleaner
(and avoids other devices starving out ISA BUS DMA devices, if those still
exist). But, either approach fixes the problem for us. So, if people object to
adding another freelist, a compromise might be renaming the ISADMA freelist to
"DMA" and bumping the region up to 32 bits.

Testing on an AMD64 VM:
# sysctl vm.phys_free
vm.phys_free:
DOMAIN 0:

FREE LIST 0:

  ORDER (SIZE)  |  NUMBER
                |  POOL 0  |  POOL 1  |  POOL 2
--            -- --      -- --      -- --      --
  12 ( 16384K)  |     257  |       0  |       0
  11 (  8192K)  |       0  |       0  |       0
  10 (  4096K)  |       2  |       1  |       0
   9 (  2048K)  |       2  |       0  |       0
   8 (  1024K)  |       1  |       0  |       0
   7 (   512K)  |       1  |       0  |       0
   6 (   256K)  |       1  |       0  |       0
   5 (   128K)  |       0  |       2  |       0
   4 (    64K)  |       0  |       1  |       0
   3 (    32K)  |       4  |       1  |       0
   2 (    16K)  |       3  |       3  |       0
   1 (     8K)  |       0  |       9  |       0
   0 (     4K)  |       1  |       1  |       0

FREE LIST 1:

  ORDER (SIZE)  |  NUMBER
                |  POOL 0  |  POOL 1  |  POOL 2
--            -- --      -- --      -- --      --
  12 ( 16384K)  |     221  |       0  |       0
  11 (  8192K)  |       1  |       0  |       0
  10 (  4096K)  |       0  |       0  |       0
   9 (  2048K)  |       1  |       0  |       0
   8 (  1024K)  |       0  |       0  |       0
   7 (   512K)  |       1  |       0  |       0
   6 (   256K)  |       1  |       0  |       0
   5 (   128K)  |       1  |       0  |       0
   4 (    64K)  |       1  |       0  |       0
   3 (    32K)  |       0  |       0  |       0
   2 (    16K)  |       0  |       0  |       0
   1 (     8K)  |       0  |       0  |       0
   0 (     4K)  |       1  |       0  |       0

FREE LIST 2:

  ORDER (SIZE)  |  NUMBER
                |  POOL 0  |  POOL 1  |  POOL 2
--            -- --      -- --      -- --      --
  12 ( 16384K)  |       0  |       0  |       0
  11 (  8192K)  |       0  |       0  |       0
  10 (  4096K)  |       0  |       0  |       0
   9 (  2048K)  |       0  |       0  |       0
   8 (  1024K)  |       1  |       0  |       0
   7 (   512K)  |       0  |       0  |       0
   6 (   256K)  |       1  |       0  |       0
   5 (   128K)  |       1  |       0  |       0
   4 (    64K)  |       2  |       0  |       0
   3 (    32K)  |       2  |       0  |       0
   2 (    16K)  |       1  |       0  |       0
   1 (     8K)  |       0  |       0  |       0
   0 (     4K)  |       1  |       0  |       0

-- 
You are receiving this mail because:
You are the assignee for the bug.


More information about the freebsd-bugs mailing list