svn commit: r356833 - in stable/12/sys: arm/arm arm/conf arm/include conf kern sys
Ian Lepore
ian at FreeBSD.org
Fri Jan 17 15:55:17 UTC 2020
Author: ian
Date: Fri Jan 17 15:55:14 2020
New Revision: 356833
URL: https://svnweb.freebsd.org/changeset/base/356833
Log:
MFC r354709-r354710, r355069, r355439, r355780, r356273, r356472
r354709:
Rewrite arm/stack_machdep.c for EABI; add stack(9) support to arm kernels.
The old stack_machdep.c code was written for the APCS ABI (aka "oldabi").
When we switched to ARM EABI (back in freebsd 10) this file never got
updated, and apparently nobody noticed that until now.
The new implementation uses the same stack unwinder code used by the
arm implemenation of the db_trace stuff.
r354710:
Compile in arm/unwind.c if options STACK is in effect; the new arm stack(9)
code now uses unwind.c.
r355069:
When doing ARM stack unwinding as part of stack_save(9), do not search
loaded modules (pass 0/false for the can_lock arg). Searching the unwind
info in modules acquires an exclusive sxlock, and the stack(9) functions can
be called in a context where unbounded sleeps are forbidden (such as from
the witness checkorder code).
Just ignoring the existence of modules in stack_save() is not ideal, so I'm
looking for a better solution, but this commit will make it possible to boot
an ARM kernel with WITNESS enabled again, until I get something better.
PR: 242200
r355439:
Declare the global kernel symbols created by ldscript.arm in arm's machdep.h,
and remove a couple scattered local declarations.
Most of these aren't referenced in C code (there are some references in
asm code), and they also aren't documented anywhere. This helps a bit
with the latter.
r355780:
Rewrite arm kernel stack unwind code to work when unwinding through modules.
The arm kernel stack unwinder has apparently never been able to unwind when
the path of execution leads through a kernel module. There was code that
tried to handle modules by looking for the unwind data in them, but it did
so by trying to find symbols which have never existed in arm kernel
modules. That caused the unwind code to panic, and because part of panic
handling calls into the unwind code, that just created a recursion loop.
Locating the unwind data in a loaded module requires accessing the Elf
section headers to find the SHT_ARM_EXIDX section. For preloaded modules
those headers are present in a metadata blob. For dynamically loaded
modules, the headers are present only while the loading is in progress; the
memory is freed once the module is ready to use. For that reason, there is
new code in kern/link_elf.c, wrapped in #ifdef __arm__, to extract the
unwind info while the headers are loaded. The values are saved into new
fields in the linker_file structure which are also conditional on __arm__.
In arm/unwind.c there is new code to locally cache the per-module info
needed to find the unwind tables. The local cache is crafted for lockless
read access, because the unwind code often needs to run in context where
sleeping is not allowed. A large comment block describes the local cache
list, so I won't repeat it all here.
r356273:
Since arm/unwind.c s conditionally compiled, only call functions in it
when one of those conditions is true. Fixes build failure on kernel
configs with no debugging options active.
r356472:
Add #ifdef option-test wrappers around another call to an arm/unwind.c
function which is only compiled-in with certain options.
Why is it always the most trivial part of a big commit that takes 3 tries
to get right?
Modified:
stable/12/sys/arm/arm/elf_machdep.c
stable/12/sys/arm/arm/mp_machdep.c
stable/12/sys/arm/arm/stack_machdep.c
stable/12/sys/arm/arm/unwind.c
stable/12/sys/arm/conf/std.armv6
stable/12/sys/arm/conf/std.armv7
stable/12/sys/arm/include/machdep.h
stable/12/sys/arm/include/stack.h
stable/12/sys/conf/files.arm
stable/12/sys/kern/kern_linker.c
stable/12/sys/kern/link_elf.c
stable/12/sys/sys/linker.h
Directory Properties:
stable/12/ (props changed)
Modified: stable/12/sys/arm/arm/elf_machdep.c
==============================================================================
--- stable/12/sys/arm/arm/elf_machdep.c Fri Jan 17 15:45:39 2020 (r356832)
+++ stable/12/sys/arm/arm/elf_machdep.c Fri Jan 17 15:55:14 2020 (r356833)
@@ -47,10 +47,15 @@ __FBSDID("$FreeBSD$");
#include <machine/elf.h>
#include <machine/md_var.h>
+#include <machine/stack.h>
#ifdef VFP
#include <machine/vfp.h>
#endif
+#include "opt_ddb.h" /* for OPT_DDB */
+#include "opt_global.h" /* for OPT_KDTRACE_HOOKS */
+#include "opt_stack.h" /* for OPT_STACK */
+
static boolean_t elf32_arm_abi_supported(struct image_params *);
u_long elf_hwcap;
@@ -309,12 +314,25 @@ elf_cpu_load_file(linker_file_t lf)
cpu_l2cache_wb_range((vm_offset_t)lf->address, (vm_size_t)lf->size);
cpu_icache_sync_range((vm_offset_t)lf->address, (vm_size_t)lf->size);
#endif
+
+#if defined(DDB) || defined(KDTRACE_HOOKS) || defined(STACK)
+ /*
+ * Inform the stack(9) code of the new module, so it can acquire its
+ * per-module unwind data.
+ */
+ unwind_module_loaded(lf);
+#endif
+
return (0);
}
int
-elf_cpu_unload_file(linker_file_t lf __unused)
+elf_cpu_unload_file(linker_file_t lf)
{
+#if defined(DDB) || defined(KDTRACE_HOOKS) || defined(STACK)
+ /* Inform the stack(9) code that this module is gone. */
+ unwind_module_unloaded(lf);
+#endif
return (0);
}
Modified: stable/12/sys/arm/arm/mp_machdep.c
==============================================================================
--- stable/12/sys/arm/arm/mp_machdep.c Fri Jan 17 15:45:39 2020 (r356832)
+++ stable/12/sys/arm/arm/mp_machdep.c Fri Jan 17 15:55:14 2020 (r356833)
@@ -106,8 +106,6 @@ check_ap(void)
return (-2);
}
-extern unsigned char _end[];
-
/* Initialize and fire up non-boot processors */
void
cpu_mp_start(void)
Modified: stable/12/sys/arm/arm/stack_machdep.c
==============================================================================
--- stable/12/sys/arm/arm/stack_machdep.c Fri Jan 17 15:45:39 2020 (r356832)
+++ stable/12/sys/arm/arm/stack_machdep.c Fri Jan 17 15:55:14 2020 (r356833)
@@ -1,8 +1,7 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
+ * SPDX-License-Identifier: BSD-2-Clause
*
- * Copyright (c) 2005 Antoine Brodin
- * All rights reserved.
+ * Copyright (c) 2019 Ian Lepore <ian at freebsd.org>
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -29,63 +28,64 @@
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
-#include <sys/param.h>
+#include <sys/types.h>
#include <sys/systm.h>
#include <sys/proc.h>
#include <sys/stack.h>
-
-#include <machine/vmparam.h>
#include <machine/pcb.h>
#include <machine/stack.h>
-/*
- * This code makes assumptions about the stack layout. These are correct
- * when using APCS (the old ABI), but are no longer true with AAPCS and the
- * ARM EABI. There is also an issue with clang and llvm when building for
- * APCS where it lays out the stack incorrectly. Because of this we disable
- * this when building for ARM EABI or when building with clang.
- */
+static void
+stack_capture(struct stack *st, struct unwind_state *state)
+{
-extern vm_offset_t kernel_vm_end;
+ stack_zero(st);
+ while (unwind_stack_one(state, 0) == 0) {
+ if (stack_put(st, state->registers[PC]) == -1)
+ break;
+ }
+}
-static void
-stack_capture(struct stack *st, u_int32_t *frame)
+void
+stack_save(struct stack *st)
{
+ struct unwind_state state;
+ uint32_t sp;
+
+ /* Read the stack pointer */
+ __asm __volatile("mov %0, sp" : "=&r" (sp));
+
+ state.registers[FP] = (uint32_t)__builtin_frame_address(0);
+ state.registers[SP] = sp;
+ state.registers[LR] = (uint32_t)__builtin_return_address(0);
+ state.registers[PC] = (uint32_t)stack_save;
+
+ stack_capture(st, &state);
}
void
stack_save_td(struct stack *st, struct thread *td)
{
- u_int32_t *frame;
+ struct unwind_state state;
- if (TD_IS_SWAPPED(td))
- panic("stack_save_td: swapped");
- if (TD_IS_RUNNING(td))
- panic("stack_save_td: running");
+ KASSERT(!TD_IS_SWAPPED(td), ("stack_save_td: swapped"));
+ KASSERT(!TD_IS_RUNNING(td), ("stack_save_td: running"));
- /*
- * This register, the frame pointer, is incorrect for the ARM EABI
- * as it doesn't have a frame pointer, however it's value is not used
- * when building for EABI.
- */
- frame = (u_int32_t *)td->td_pcb->pcb_regs.sf_r11;
- stack_zero(st);
- stack_capture(st, frame);
+ state.registers[FP] = td->td_pcb->pcb_regs.sf_r11;
+ state.registers[SP] = td->td_pcb->pcb_regs.sf_sp;
+ state.registers[LR] = td->td_pcb->pcb_regs.sf_lr;
+ state.registers[PC] = td->td_pcb->pcb_regs.sf_pc;
+
+ stack_capture(st, &state);
}
int
stack_save_td_running(struct stack *st, struct thread *td)
{
+ if (td == curthread) {
+ stack_save(st);
+ return (0);
+ }
return (EOPNOTSUPP);
-}
-
-void
-stack_save(struct stack *st)
-{
- u_int32_t *frame;
-
- frame = (u_int32_t *)__builtin_frame_address(0);
- stack_zero(st);
- stack_capture(st, frame);
}
Modified: stable/12/sys/arm/arm/unwind.c
==============================================================================
--- stable/12/sys/arm/arm/unwind.c Fri Jan 17 15:45:39 2020 (r356832)
+++ stable/12/sys/arm/arm/unwind.c Fri Jan 17 15:55:14 2020 (r356833)
@@ -32,9 +32,13 @@
__FBSDID("$FreeBSD$");
#include <sys/param.h>
-#include <sys/systm.h>
+#include <sys/kernel.h>
#include <sys/linker.h>
+#include <sys/malloc.h>
+#include <sys/queue.h>
+#include <sys/systm.h>
+#include <machine/machdep.h>
#include <machine/stack.h>
#include "linker_if.h"
@@ -61,12 +65,6 @@ __FBSDID("$FreeBSD$");
#define EXIDX_CANTUNWIND 1
/*
- * These are set in the linker script. Their addresses will be
- * either the start or end of the exception table or index.
- */
-extern int exidx_start, exidx_end;
-
-/*
* Entry types.
* These are the only entry types that have been seen in the kernel.
*/
@@ -99,79 +97,236 @@ struct unwind_idx {
uint32_t insn;
};
-/* Expand a 31-bit signed value to a 32-bit signed value */
-static __inline int32_t
-expand_prel31(uint32_t prel31)
+/*
+ * Local cache of unwind info for loaded modules.
+ *
+ * To unwind the stack through the code in a loaded module, we need to access
+ * the module's exidx unwind data. To locate that data, one must search the
+ * elf section headers for the SHT_ARM_EXIDX section. Those headers are
+ * available at the time the module is being loaded, but are discarded by time
+ * the load process has completed. Code in kern/link_elf.c locates the data we
+ * need and stores it into the linker_file structure before calling the arm
+ * machdep routine for handling loaded modules (in arm/elf_machdep.c). That
+ * function calls into this code to pass along the unwind info, which we save
+ * into one of these module_info structures.
+ *
+ * Because we have to help stack(9) gather stack info at any time, including in
+ * contexts where sleeping is not allowed, we cannot use linker_file_foreach()
+ * to walk the kernel's list of linker_file structs, because doing so requires
+ * acquiring an exclusive sx_lock. So instead, we keep a local list of these
+ * structures, one for each loaded module (and one for the kernel itself that we
+ * synthesize at init time). New entries are added to the end of this list as
+ * needed, but entries are never deleted from the list. Instead, they are
+ * cleared out in-place to mark them as unused. That means the code doing stack
+ * unwinding can always safely walk the list without locking, because the
+ * structure of the list never changes in a way that would cause the walker to
+ * follow a bad link.
+ *
+ * A cleared-out entry on the list has module start=UINTPTR_MAX and end=0, so
+ * start <= addr < end cannot be true for any value of addr being searched for.
+ * We also don't have to worry about races where we look up the unwind info just
+ * before a module is unloaded and try to access it concurrently with or just
+ * after the unloading happens in another thread, because that means the path of
+ * execution leads through a now-unloaded module, and that's already well into
+ * undefined-behavior territory.
+ *
+ * List entries marked as unused get reused when new modules are loaded. We
+ * don't worry about holding a few unused bytes of memory in the list after
+ * unloading a module.
+ */
+struct module_info {
+ uintptr_t module_start; /* Start of loaded module */
+ uintptr_t module_end; /* End of loaded module */
+ uintptr_t exidx_start; /* Start of unwind data */
+ uintptr_t exidx_end; /* End of unwind data */
+ STAILQ_ENTRY(module_info)
+ link; /* Link to next entry */
+};
+static STAILQ_HEAD(, module_info) module_list;
+
+/*
+ * Hide ugly casting in somewhat-less-ugly macros.
+ * CADDR - cast a pointer or number to caddr_t.
+ * UADDR - cast a pointer or number to uintptr_t.
+ */
+#define CADDR(addr) ((caddr_t)(void*)(uintptr_t)(addr))
+#define UADDR(addr) ((uintptr_t)(addr))
+
+/*
+ * Clear the info in an existing module_info entry on the list. The
+ * module_start/end addresses are set to values that cannot match any real
+ * memory address. The entry remains on the list, but will be ignored until it
+ * is populated with new data.
+ */
+static void
+clear_module_info(struct module_info *info)
{
+ info->module_start = UINTPTR_MAX;
+ info->module_end = 0;
+}
- return ((int32_t)(prel31 & 0x7fffffffu) << 1) / 2;
+/*
+ * Populate an existing module_info entry (which is already on the list) with
+ * the info for a new module.
+ */
+static void
+populate_module_info(struct module_info *info, linker_file_t lf)
+{
+
+ /*
+ * Careful! The module_start and module_end fields must not be set
+ * until all other data in the structure is valid.
+ */
+ info->exidx_start = UADDR(lf->exidx_addr);
+ info->exidx_end = UADDR(lf->exidx_addr) + lf->exidx_size;
+ info->module_start = UADDR(lf->address);
+ info->module_end = UADDR(lf->address) + lf->size;
}
-struct search_context {
- uint32_t addr;
- caddr_t exidx_start;
- caddr_t exidx_end;
-};
+/*
+ * Create a new empty module_info entry and add it to the tail of the list.
+ */
+static struct module_info *
+create_module_info(void)
+{
+ struct module_info *info;
-static int
-module_search(linker_file_t lf, void *context)
+ info = malloc(sizeof(*info), M_CACHE, M_WAITOK | M_ZERO);
+ clear_module_info(info);
+ STAILQ_INSERT_TAIL(&module_list, info, link);
+ return (info);
+}
+
+/*
+ * Search for a module_info entry on the list whose address range contains the
+ * given address. If the search address is zero (no module will be loaded at
+ * zero), then we're looking for an empty item to reuse, which is indicated by
+ * module_start being set to UINTPTR_MAX in the entry.
+ */
+static struct module_info *
+find_module_info(uintptr_t addr)
{
- struct search_context *sc = context;
- linker_symval_t symval;
- c_linker_sym_t sym;
+ struct module_info *info;
- if (lf->address <= (caddr_t)sc->addr &&
- (lf->address + lf->size) >= (caddr_t)sc->addr) {
- if ((LINKER_LOOKUP_SYMBOL(lf, "__exidx_start", &sym) == 0 ||
- LINKER_LOOKUP_SYMBOL(lf, "exidx_start", &sym) == 0) &&
- LINKER_SYMBOL_VALUES(lf, sym, &symval) == 0)
- sc->exidx_start = symval.value;
+ STAILQ_FOREACH(info, &module_list, link) {
+ if ((addr >= info->module_start && addr < info->module_end) ||
+ (addr == 0 && info->module_start == UINTPTR_MAX))
+ return (info);
+ }
+ return (NULL);
+}
- if ((LINKER_LOOKUP_SYMBOL(lf, "__exidx_end", &sym) == 0 ||
- LINKER_LOOKUP_SYMBOL(lf, "exidx_end", &sym) == 0) &&
- LINKER_SYMBOL_VALUES(lf, sym, &symval) == 0)
- sc->exidx_end = symval.value;
+/*
+ * Handle the loading of a new module by populating a module_info for it. This
+ * is called for both preloaded and dynamically loaded modules.
+ */
+void
+unwind_module_loaded(struct linker_file *lf)
+{
+ struct module_info *info;
- if (sc->exidx_start != NULL && sc->exidx_end != NULL)
- return (1);
- panic("Invalid module %s, no unwind tables\n", lf->filename);
+ /*
+ * A module that contains only data may have no unwind info; don't
+ * create any module info for it.
+ */
+ if (lf->exidx_size == 0)
+ return;
+
+ /*
+ * Find an unused entry in the existing list to reuse. If we don't find
+ * one, create a new one and link it into the list. This is the only
+ * place the module_list is modified. Adding a new entry to the list
+ * will not perturb any other threads currently walking the list. This
+ * function is invoked while kern_linker is still holding its lock
+ * to prevent its module list from being modified, so we don't have to
+ * worry about racing other threads doing an insert concurrently.
+ */
+ if ((info = find_module_info(0)) == NULL) {
+ info = create_module_info();
}
+ populate_module_info(info, lf);
+}
+
+/* Handle the unloading of a module. */
+void
+unwind_module_unloaded(struct linker_file *lf)
+{
+ struct module_info *info;
+
+ /*
+ * A module that contains only data may have no unwind info and there
+ * won't be a list entry for it.
+ */
+ if (lf->exidx_size == 0)
+ return;
+
+ /*
+ * When a module is unloaded, we clear the info out of its entry in the
+ * module list, making that entry available for later reuse.
+ */
+ if ((info = find_module_info(UADDR(lf->address))) == NULL) {
+ printf("arm unwind: module '%s' not on list at unload time\n",
+ lf->filename);
+ return;
+ }
+ clear_module_info(info);
+}
+
+/*
+ * Initialization must run fairly early, as soon as malloc(9) is available, and
+ * definitely before witness, which uses stack(9). We synthesize a module_info
+ * entry for the kernel, because unwind_module_loaded() doesn't get called for
+ * it. Also, it is unlike other modules in that the elf metadata for locating
+ * the unwind tables might be stripped, so instead we have to use the
+ * _exidx_start/end symbols created by ldscript.arm.
+ */
+static int
+module_info_init(void *arg __unused)
+{
+ struct linker_file thekernel;
+
+ STAILQ_INIT(&module_list);
+
+ thekernel.filename = "kernel";
+ thekernel.address = CADDR(&_start);
+ thekernel.size = UADDR(&_end) - UADDR(&_start);
+ thekernel.exidx_addr = CADDR(&_exidx_start);
+ thekernel.exidx_size = UADDR(&_exidx_end) - UADDR(&_exidx_start);
+ populate_module_info(create_module_info(), &thekernel);
+
return (0);
}
+SYSINIT(unwind_init, SI_SUB_KMEM, SI_ORDER_ANY, module_info_init, NULL);
+/* Expand a 31-bit signed value to a 32-bit signed value */
+static __inline int32_t
+expand_prel31(uint32_t prel31)
+{
+
+ return ((int32_t)(prel31 & 0x7fffffffu) << 1) / 2;
+}
+
/*
* Perform a binary search of the index table to find the function
* with the largest address that doesn't exceed addr.
*/
static struct unwind_idx *
-find_index(uint32_t addr, int search_modules)
+find_index(uint32_t addr)
{
- struct search_context sc;
- caddr_t idx_start, idx_end;
+ struct module_info *info;
unsigned int min, mid, max;
struct unwind_idx *start;
struct unwind_idx *item;
int32_t prel31_addr;
uint32_t func_addr;
- start = (struct unwind_idx *)&exidx_start;
- idx_start = (caddr_t)&exidx_start;
- idx_end = (caddr_t)&exidx_end;
+ info = find_module_info(addr);
+ if (info == NULL)
+ return NULL;
- /* This may acquire a lock */
- if (search_modules) {
- bzero(&sc, sizeof(sc));
- sc.addr = addr;
- if (linker_file_foreach(module_search, &sc) != 0 &&
- sc.exidx_start != NULL && sc.exidx_end != NULL) {
- start = (struct unwind_idx *)sc.exidx_start;
- idx_start = sc.exidx_start;
- idx_end = sc.exidx_end;
- }
- }
-
min = 0;
- max = (idx_end - idx_start) / sizeof(struct unwind_idx);
+ max = (info->exidx_end - info->exidx_start) / sizeof(struct unwind_idx);
+ start = (struct unwind_idx *)CADDR(info->exidx_start);
while (min != max) {
mid = min + (max - min + 1) / 2;
@@ -382,11 +537,17 @@ unwind_tab(struct unwind_state *state)
return 0;
}
+/*
+ * Unwind a single stack frame.
+ * Return 0 on success or 1 if the stack cannot be unwound any further.
+ *
+ * XXX The can_lock argument is no longer germane; a sweep of callers should be
+ * made to remove it after this new code has proven itself for a while.
+ */
int
-unwind_stack_one(struct unwind_state *state, int can_lock)
+unwind_stack_one(struct unwind_state *state, int can_lock __unused)
{
struct unwind_idx *index;
- int finished;
/* Reset the mask of updated registers */
state->update_mask = 0;
@@ -395,26 +556,20 @@ unwind_stack_one(struct unwind_state *state, int can_l
state->start_pc = state->registers[PC];
/* Find the item to run */
- index = find_index(state->start_pc, can_lock);
+ index = find_index(state->start_pc);
+ if (index == NULL || index->insn == EXIDX_CANTUNWIND)
+ return 1;
- finished = 0;
- if (index->insn != EXIDX_CANTUNWIND) {
- if (index->insn & (1U << 31)) {
- /* The data is within the instruction */
- state->insn = &index->insn;
- } else {
- /* A prel31 offset to the unwind table */
- state->insn = (uint32_t *)
- ((uintptr_t)&index->insn +
- expand_prel31(index->insn));
- }
- /* Run the unwind function */
- finished = unwind_tab(state);
+ if (index->insn & (1U << 31)) {
+ /* The data is within the instruction */
+ state->insn = &index->insn;
+ } else {
+ /* A prel31 offset to the unwind table */
+ state->insn = (uint32_t *)
+ ((uintptr_t)&index->insn +
+ expand_prel31(index->insn));
}
- /* This is the top of the stack, finish */
- if (index->insn == EXIDX_CANTUNWIND)
- finished = 1;
-
- return (finished);
+ /* Run the unwind function, return its finished/not-finished status. */
+ return (unwind_tab(state));
}
Modified: stable/12/sys/arm/conf/std.armv6
==============================================================================
--- stable/12/sys/arm/conf/std.armv6 Fri Jan 17 15:45:39 2020 (r356832)
+++ stable/12/sys/arm/conf/std.armv6 Fri Jan 17 15:55:14 2020 (r356833)
@@ -34,6 +34,7 @@ options GEOM_LABEL # Provides labelization
options COMPAT_43 # Compatible with BSD 4.3 [KEEP THIS!]
options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI
options KTRACE # ktrace(1) support
+options STACK # stack(9) support
options SYSVSHM # SYSV-style shared memory
options SYSVMSG # SYSV-style message queues
options SYSVSEM # SYSV-style semaphores
Modified: stable/12/sys/arm/conf/std.armv7
==============================================================================
--- stable/12/sys/arm/conf/std.armv7 Fri Jan 17 15:45:39 2020 (r356832)
+++ stable/12/sys/arm/conf/std.armv7 Fri Jan 17 15:55:14 2020 (r356833)
@@ -34,6 +34,7 @@ options GEOM_LABEL # Provides labelization
options COMPAT_43 # Compatible with BSD 4.3 [KEEP THIS!]
options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI
options KTRACE # ktrace(1) support
+options STACK # stack(9) support
options SYSVSHM # SYSV-style shared memory
options SYSVMSG # SYSV-style message queues
options SYSVSEM # SYSV-style semaphores
Modified: stable/12/sys/arm/include/machdep.h
==============================================================================
--- stable/12/sys/arm/include/machdep.h Fri Jan 17 15:45:39 2020 (r356832)
+++ stable/12/sys/arm/include/machdep.h Fri Jan 17 15:55:14 2020 (r356833)
@@ -61,4 +61,24 @@ void arm_add_efi_map_entries(struct efi_map_header *ef
struct mem_region *mr, int *mrcnt);
#endif
+/*
+ * Symbols created by ldscript.arm which are accessible in the kernel as global
+ * symbols. They have uint8 type because they mark the byte location where the
+ * corresponding data starts or ends (in the end case, it's the next byte
+ * following the data, so the data size is end-start). These are listed below
+ * in the order they occur within the kernel (i.e., the address of each variable
+ * should be greater than any of the ones before it).
+ */
+extern uint8_t _start; /* Kernel entry point in locore.S */
+extern uint8_t _etext; /* text segment end */
+extern uint8_t _extab_start; /* unwind table start */
+extern uint8_t _exidx_start; /* unwind index start */
+extern uint8_t _exidx_end; /* unwind index end */
+extern uint8_t _start_ctors; /* ctors data start */
+extern uint8_t _stop_ctors; /* ctors data end */
+extern uint8_t _edata; /* data segment end */
+extern uint8_t __bss_start; /* bss segment start */
+extern uint8_t _ebss; /* bss segment end */
+extern uint8_t _end; /* End of kernel (text+ctors+unwind+data+bss) */
+
#endif /* !_MACHINE_MACHDEP_H_ */
Modified: stable/12/sys/arm/include/stack.h
==============================================================================
--- stable/12/sys/arm/include/stack.h Fri Jan 17 15:45:39 2020 (r356832)
+++ stable/12/sys/arm/include/stack.h Fri Jan 17 15:55:14 2020 (r356833)
@@ -55,6 +55,14 @@ struct unwind_state {
#define LR 14
#define PC 15
+#ifdef _KERNEL
+
int unwind_stack_one(struct unwind_state *, int);
+
+struct linker_file;
+void unwind_module_loaded(struct linker_file *);
+void unwind_module_unloaded(struct linker_file *);
+
+#endif
#endif /* !_MACHINE_STACK_H_ */
Modified: stable/12/sys/conf/files.arm
==============================================================================
--- stable/12/sys/conf/files.arm Fri Jan 17 15:45:39 2020 (r356832)
+++ stable/12/sys/conf/files.arm Fri Jan 17 15:55:14 2020 (r356833)
@@ -85,7 +85,7 @@ arm/arm/trap-v4.c optional !armv7 !armv6
arm/arm/trap-v6.c optional armv7 | armv6
arm/arm/uio_machdep.c standard
arm/arm/undefined.c standard
-arm/arm/unwind.c optional ddb | kdtrace_hooks
+arm/arm/unwind.c optional ddb | kdtrace_hooks | stack
arm/arm/vm_machdep.c standard
arm/arm/vfp.c standard
arm/cloudabi32/cloudabi32_sysvec.c optional compat_cloudabi32
Modified: stable/12/sys/kern/kern_linker.c
==============================================================================
--- stable/12/sys/kern/kern_linker.c Fri Jan 17 15:45:39 2020 (r356832)
+++ stable/12/sys/kern/kern_linker.c Fri Jan 17 15:55:14 2020 (r356833)
@@ -624,6 +624,10 @@ linker_make_file(const char *pathname, linker_class_t
lf->ndeps = 0;
lf->deps = NULL;
lf->loadcnt = ++loadcnt;
+#ifdef __arm__
+ lf->exidx_addr = 0;
+ lf->exidx_size = 0;
+#endif
STAILQ_INIT(&lf->common);
TAILQ_INIT(&lf->modules);
TAILQ_INSERT_TAIL(&linker_files, lf, link);
Modified: stable/12/sys/kern/link_elf.c
==============================================================================
--- stable/12/sys/kern/link_elf.c Fri Jan 17 15:45:39 2020 (r356832)
+++ stable/12/sys/kern/link_elf.c Fri Jan 17 15:55:14 2020 (r356833)
@@ -692,6 +692,50 @@ parse_vnet(elf_file_t ef)
}
#endif
+#ifdef __arm__
+/*
+ * Locate the ARM exception/unwind table info for DDB and stack(9) use by
+ * searching for the section header that describes it. There may be no unwind
+ * info, for example in a module containing only data.
+ */
+static void
+link_elf_locate_exidx(linker_file_t lf, Elf_Shdr *shdr, int nhdr)
+{
+ int i;
+
+ for (i = 0; i < nhdr; i++) {
+ if (shdr[i].sh_type == SHT_ARM_EXIDX) {
+ lf->exidx_addr = shdr[i].sh_addr + lf->address;
+ lf->exidx_size = shdr[i].sh_size;
+ break;
+ }
+ }
+}
+
+/*
+ * Locate the section headers metadata in a preloaded module, then use it to
+ * locate the exception/unwind table in the module. The size of the metadata
+ * block is stored in a uint32 word immediately before the data itself, and a
+ * comment in preload_search_info() says it is safe to rely on that.
+ */
+static void
+link_elf_locate_exidx_preload(struct linker_file *lf, caddr_t modptr)
+{
+ uint32_t *modinfo;
+ Elf_Shdr *shdr;
+ uint32_t nhdr;
+
+ modinfo = (uint32_t *)preload_search_info(modptr,
+ MODINFO_METADATA | MODINFOMD_SHDR);
+ if (modinfo != NULL) {
+ shdr = (Elf_Shdr *)modinfo;
+ nhdr = modinfo[-1] / sizeof(Elf_Shdr);
+ link_elf_locate_exidx(lf, shdr, nhdr);
+ }
+}
+
+#endif /* __arm__ */
+
static int
link_elf_link_preload(linker_class_t cls,
const char* filename, linker_file_t *result)
@@ -747,6 +791,10 @@ link_elf_link_preload(linker_class_t cls,
lf->ctors_size = *ctors_sizep;
}
+#ifdef __arm__
+ link_elf_locate_exidx_preload(lf, modptr);
+#endif
+
error = parse_dynamic(ef);
if (error == 0)
error = parse_dpcpu(ef);
@@ -1140,6 +1188,11 @@ link_elf_load_file(linker_class_t cls, const char* fil
ef->ddbstrtab = ef->strbase;
nosyms:
+
+#ifdef __arm__
+ link_elf_locate_exidx(lf, shdr, hdr->e_shnum);
+#endif
+
error = link_elf_link_common_finish(lf);
if (error != 0)
goto out;
Modified: stable/12/sys/sys/linker.h
==============================================================================
--- stable/12/sys/sys/linker.h Fri Jan 17 15:45:39 2020 (r356832)
+++ stable/12/sys/sys/linker.h Fri Jan 17 15:55:14 2020 (r356833)
@@ -97,6 +97,11 @@ struct linker_file {
*/
int nenabled; /* number of enabled probes. */
int fbt_nentries; /* number of fbt entries created. */
+
+#ifdef __arm__
+ caddr_t exidx_addr; /* Unwind data index table start */
+ size_t exidx_size; /* Unwind data index table size */
+#endif
};
/*
More information about the svn-src-all
mailing list