[Bug 192249] New: [PATCH] Load reloc modules (amd64, mips) faster by caching global syms
bugzilla-noreply at freebsd.org
bugzilla-noreply at freebsd.org
Tue Jul 29 19:35:46 UTC 2014
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192249
Bug ID: 192249
Summary: [PATCH] Load reloc modules (amd64, mips) faster by
caching global syms
Product: Base System
Version: 11.0-CURRENT
Hardware: Any
OS: Any
Status: Needs Triage
Severity: Affects Only Me
Priority: ---
Component: kern
Assignee: freebsd-bugs at FreeBSD.org
Reporter: conrad.meyer at isilon.com
Created attachment 145126
--> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=145126&action=edit
(cd src/ && patch -p1 ; diff is against recent-ish CURRENT)
The problem:
At Isilon, efs.ko is a very large kernel module; we were seeing kldload take
12-13 seconds. Most of that time was spent repeatedly resolving the same global
symbols for relocations. link_elf_obj would lookup the same globals many times;
each time, it would lookup in the local module first, before missing and
falling back to deps, where it would find 'kernel' and get a fast hit (hash
table hit). Since ld(1) doesn't generate lookup hashes (SysV or GNU) for
relocatable objects (kmods on AMD64), lookups in the local module involved
walking the entire symtab and strcmp()ing. efs.ko has 32k symbols. This was
very slow.
Solution:
Add a special ELF SHN_ value (in the SHN_LOOS to SHN_HIOS range) to represent
cached SHN_UNDEF globals. Cache the results of global lookup during relocatable
object load.
Modify Dtrace to ignore these SHN_FBSD_CACHED symbols the same way it ignored
SHN_UNDEF symbols before. I didn't find other users of SHN_UNDEF that might be
negatively affected by this change.
Testing done:
Basic Dtrace testing suggests it isn't broken. Performance results:
Before
# for i in 1 2 3 ; do time kldload efs.ko ; kldunload efs.ko ; done
kldload efs.ko 0.00s user 13.00s system 103% cpu 12.607 total
kldload efs.ko 0.00s user 13.25s system 102% cpu 12.876 total
kldload efs.ko 0.00s user 13.14s system 103% cpu 12.752 total
kldload zfs.ko 0.00s user 1.77s system 102% cpu 1.729 total # Smaller, for
comparison
After
kldload efs.ko 0.00s user 0.25s system 99% cpu 0.252 total
kldload efs.ko 0.00s user 0.20s system 99% cpu 0.204 total
kldload efs.ko 0.00s user 0.21s system 99% cpu 0.211 total
kldload zfs.ko 0.00s user 0.04s system 96% cpu 0.040 total # For
comparison
Sponsored by: EMC / Isilon storage division
--
You are receiving this mail because:
You are the assignee for the bug.
More information about the freebsd-bugs
mailing list