[Bug 192249] New: [PATCH] Load reloc modules (amd64, mips) faster by caching global syms

bugzilla-noreply at freebsd.org bugzilla-noreply at freebsd.org
Tue Jul 29 19:35:46 UTC 2014


https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192249

            Bug ID: 192249
           Summary: [PATCH] Load reloc modules (amd64, mips) faster by
                    caching global syms
           Product: Base System
           Version: 11.0-CURRENT
          Hardware: Any
                OS: Any
            Status: Needs Triage
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: freebsd-bugs at FreeBSD.org
          Reporter: conrad.meyer at isilon.com

Created attachment 145126
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=145126&action=edit
(cd src/ && patch -p1 ; diff is against recent-ish CURRENT)

The problem:

At Isilon, efs.ko is a very large kernel module; we were seeing kldload take
12-13 seconds. Most of that time was spent repeatedly resolving the same global
symbols for relocations. link_elf_obj would lookup the same globals many times;
each time, it would lookup in the local module first, before missing and
falling back to deps, where it would find 'kernel' and get a fast hit (hash
table hit). Since ld(1) doesn't generate lookup hashes (SysV or GNU) for
relocatable objects (kmods on AMD64), lookups in the local module involved
walking the entire symtab and strcmp()ing. efs.ko has 32k symbols. This was
very slow.

Solution:

Add a special ELF SHN_ value (in the SHN_LOOS to SHN_HIOS range) to represent
cached SHN_UNDEF globals. Cache the results of global lookup during relocatable
object load.

Modify Dtrace to ignore these SHN_FBSD_CACHED symbols the same way it ignored
SHN_UNDEF symbols before. I didn't find other users of SHN_UNDEF that might be
negatively affected by this change.

Testing done:

Basic Dtrace testing suggests it isn't broken. Performance results:

Before
# for i in 1 2 3 ; do time kldload efs.ko ; kldunload efs.ko ; done
kldload efs.ko  0.00s user 13.00s system 103% cpu 12.607 total
kldload efs.ko  0.00s user 13.25s system 102% cpu 12.876 total
kldload efs.ko  0.00s user 13.14s system 103% cpu 12.752 total

kldload zfs.ko  0.00s user 1.77s system 102% cpu 1.729 total    # Smaller, for
comparison

After
kldload efs.ko  0.00s user 0.25s system 99% cpu 0.252 total
kldload efs.ko  0.00s user 0.20s system 99% cpu 0.204 total
kldload efs.ko  0.00s user 0.21s system 99% cpu 0.211 total

kldload zfs.ko  0.00s user 0.04s system 96% cpu 0.040 total     # For
comparison


Sponsored by:    EMC / Isilon storage division

-- 
You are receiving this mail because:
You are the assignee for the bug.


More information about the freebsd-bugs mailing list