From nobody Sun Jan 09 01:28:35 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 7EEC9193F645; Sun, 9 Jan 2022 01:28:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JWfVX2gy9z3MBs; Sun, 9 Jan 2022 01:28:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C17B411B9; Sun, 9 Jan 2022 01:28:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 2091SZLr002060; Sun, 9 Jan 2022 01:28:35 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 2091SZGG002059; Sun, 9 Jan 2022 01:28:35 GMT (envelope-from git) Date: Sun, 9 Jan 2022 01:28:35 GMT Message-Id: <202201090128.2091SZGG002059@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: f131d68e21fc - stable/12 - efi: loader: Inline copy_finish function in amd64 trampoline List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: f131d68e21fc360f5c8e63377f25cf60706d9afa Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1641691716; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=a8GUQuq/5W927kgudvFWHjO7gFMDEfY7P41ti+xt9F0=; b=JdwjJQIUrzYtj9VSEHOzhkczqslLjRJUSWFrEL550JK/O3aa3ylA76m6lHW/yFLODnB3wE eEGz/pC0cyWKaTWMI741WiMVRnuSsn+J2hXK6zBAnS7jEG9YuMlo53WVGG8y6gTntgt3mm jKdiH2aJY9Z8QclNPnVNqgMq4mt80n7RUI9Ul+J335tE1FhIUHEL1Y6MwjUcjEOlK6q2Tg bmBsSkQXBzDqpN3rX+oFbJqNl+NaAOPbPuUWj/ZPmjBqt9yJChExLquYte8PdACnKnkHTu BxKj0El02pcvuNTP6MQ/oa5O2neRpWILZBQCf6Xkoei6B5tPmUfIPSCZtJNNgg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1641691716; a=rsa-sha256; cv=none; b=rNnae1iI6dFkCB5ziQC/Xpt572G/reOJIQS5fPNJfdS2WrWPeu554p3dAgOJgpClzhR0GG 9/LTonOYLpm0IaoP0wUaVKMZ6M5UXqSaRxZO2d2IWAVDcIxD1HkHk4e4Mz/6qBPVu6aLMd H6nY0NIhrVvvMu6j4KvXPsza1G7cwUF3Tq6zoNinaw8ZwPDDxduDr96f6ZXiGCgGBNi8PG MBxXK10U+XjEIauN2vz6CY5AJWNOzUKqit+ZS2+yFeblqaIOKm54jpDhH9DDCKVA2yup/c fBI6pFIo2iga+qe009nbmiDU80+6wG4xjHlaog1RCxo1icd0qCkZ2u/W7+Iflg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/12 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=f131d68e21fc360f5c8e63377f25cf60706d9afa commit f131d68e21fc360f5c8e63377f25cf60706d9afa Author: David Sebek AuthorDate: 2022-01-07 20:18:49 +0000 Commit: Konstantin Belousov CommitDate: 2022-01-09 01:28:01 +0000 efi: loader: Inline copy_finish function in amd64 trampoline Instead of calling the efi_copy_finish function from amd64_tramp, include the copy instructions in the trampoline code itself. This avoids boot hangs and restarts in the cases when the efi_copy_finish code happens to be located at a memory location that is overwritten during the copy process. This is a direct commit to stable/12 since no-copy loader mode requires the kernel update not suitable for the branch. PR: 209821 --- stand/efi/loader/arch/amd64/amd64_tramp.S | 59 +++++++++++++++++++---------- stand/efi/loader/arch/amd64/elf64_freebsd.c | 18 +++++---- stand/efi/loader/copy.c | 8 ++++ stand/efi/loader/loader_efi.h | 1 + 4 files changed, 60 insertions(+), 26 deletions(-) diff --git a/stand/efi/loader/arch/amd64/amd64_tramp.S b/stand/efi/loader/arch/amd64/amd64_tramp.S index c102d9243589..3ca98a7cf38a 100644 --- a/stand/efi/loader/arch/amd64/amd64_tramp.S +++ b/stand/efi/loader/arch/amd64/amd64_tramp.S @@ -1,9 +1,11 @@ /*- * Copyright (c) 2013 The FreeBSD Foundation + * Copyright 2021 David Sebek * All rights reserved. * * This software was developed by Benno Rice under sponsorship from * the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -31,34 +33,53 @@ #include .text - .globl amd64_tramp + .globl amd64_tramp_inline /* - * void amd64_tramp(uint64_t stack, void *copy_finish, uint64_t kernend, - * uint64_t modulep, uint64_t pagetable, uint64_t entry) + * void amd64_tramp_inline(uint64_t stack %rdi, uint64_t kernend %rsi, + * uint64_t modulep %rdx, uint64_t pagetable %rcx, uint64_t entry %r8, + * uint64_t copy_dst %r9, uint64_t copy_src 8(%rsp), + * uint64_t copy_src_end 16(%rsp)) */ -amd64_tramp: +amd64_tramp_inline: cli /* Make sure we don't get interrupted. */ - movq %rdi,%rsp /* Switch to our temporary stack. */ - movq %rdx,%r12 /* Stash the kernel values for later. */ - movq %rcx,%r13 - movq %r8,%r14 - movq %r9,%r15 + /* + * Copy the kernel from the staging area to the expected location + * in memory. The following code is equivalent to the efi_copy_finish + * function that amd64_tramp used to call. Inlining this code avoids + * a scenario when the system froze because efi_copy_finish + * overwrote its own code that just happened to be located somewhere + * in the destination range. + * + * while (copy_src < copy_src_end) *copy_dst++ = *copy_src++; + */ + movq 8(%rsp), %rax /* rax = copy_src */ + movq 16(%rsp), %r10 /* r10 = copy_src_end */ + cmpq %r10, %rax + jnb copy_done + subq %rax, %r9 /* r9 = copy_dst - copy_src */ +loop: + movq (%rax), %r11 + movq %r11, (%rax,%r9) + addq $8, %rax + cmpq %rax, %r10 + ja loop +copy_done: - callq *%rsi /* Call copy_finish so we're all ready to go. */ + movq %rdi,%rsp /* Switch to our temporary stack. */ - pushq %r12 /* Push kernend. */ - salq $32,%r13 /* Shift modulep and push it. */ - pushq %r13 - pushq %r15 /* Push the entry address. */ - movq %r14,%cr3 /* Switch page tables. */ + pushq %rsi /* Push kernend. */ + salq $32,%rdx /* Shift modulep and push it. */ + pushq %rdx + pushq %r8 /* Push the entry address. */ + movq %rcx,%cr3 /* Switch page tables. */ ret /* "Return" to kernel entry. */ ALIGN_TEXT -amd64_tramp_end: +amd64_tramp_inline_end: .data - .globl amd64_tramp_size -amd64_tramp_size: - .long amd64_tramp_end-amd64_tramp + .globl amd64_tramp_inline_size +amd64_tramp_inline_size: + .long amd64_tramp_inline_end-amd64_tramp_inline diff --git a/stand/efi/loader/arch/amd64/elf64_freebsd.c b/stand/efi/loader/arch/amd64/elf64_freebsd.c index 896041e066c9..6fcd561cb2ff 100644 --- a/stand/efi/loader/arch/amd64/elf64_freebsd.c +++ b/stand/efi/loader/arch/amd64/elf64_freebsd.c @@ -78,11 +78,12 @@ static pml4_entry_t *PT4; static pdp_entry_t *PT3; static pd_entry_t *PT2; -static void (*trampoline)(uint64_t stack, void *copy_finish, uint64_t kernend, - uint64_t modulep, pml4_entry_t *pagetable, uint64_t entry); +static void (*trampoline)(uint64_t stack, uint64_t kernend, + uint64_t modulep, pml4_entry_t *pagetable, uint64_t entry, + uint64_t copy_dst, uint64_t copy_src, uint64_t copy_src_end); -extern uintptr_t amd64_tramp; -extern uint32_t amd64_tramp_size; +extern uintptr_t amd64_tramp_inline; +extern uint32_t amd64_tramp_inline_size; /* * There is an ELF kernel and one or more ELF modules loaded. @@ -95,6 +96,7 @@ elf64_exec(struct preloaded_file *fp) struct file_metadata *md; Elf_Ehdr *ehdr; vm_offset_t modulep, kernend, trampcode, trampstack; + uint64_t copy_dst, copy_src, copy_src_end; int err, i; ACPI_TABLE_RSDP *rsdp; char buf[24]; @@ -153,7 +155,7 @@ elf64_exec(struct preloaded_file *fp) (EFI_PHYSICAL_ADDRESS *)&trampcode); bzero((void *)trampcode, EFI_PAGE_SIZE); trampstack = trampcode + EFI_PAGE_SIZE - 8; - bcopy((void *)&amd64_tramp, (void *)trampcode, amd64_tramp_size); + bcopy((void *)&amd64_tramp_inline, (void *)trampcode, amd64_tramp_inline_size); trampoline = (void *)trampcode; PT4 = (pml4_entry_t *)0x0000000040000000; @@ -194,8 +196,10 @@ elf64_exec(struct preloaded_file *fp) dev_cleanup(); - trampoline(trampstack, efi_copy_finish, kernend, modulep, PT4, - ehdr->e_entry); + efi_copy_get_locations(©_dst, ©_src, ©_src_end); + + trampoline(trampstack, kernend, modulep, PT4, + ehdr->e_entry, copy_dst, copy_src, copy_src_end); panic("exec returned"); } diff --git a/stand/efi/loader/copy.c b/stand/efi/loader/copy.c index 5a174dbf51e2..307ad3e0ac79 100644 --- a/stand/efi/loader/copy.c +++ b/stand/efi/loader/copy.c @@ -361,3 +361,11 @@ efi_copy_finish(void) while (src < last) *dst++ = *src++; } + +void +efi_copy_get_locations(uint64_t *dst, uint64_t *src, uint64_t *src_end) +{ + *src = (uint64_t)staging; + *dst = (uint64_t)(staging - stage_offset); + *src_end = (uint64_t)staging_end; +} diff --git a/stand/efi/loader/loader_efi.h b/stand/efi/loader/loader_efi.h index 4d077514e423..5f0c7c6825a0 100644 --- a/stand/efi/loader/loader_efi.h +++ b/stand/efi/loader/loader_efi.h @@ -44,5 +44,6 @@ ssize_t efi_readin(readin_handle_t fd, vm_offset_t dest, const size_t len); void * efi_translate(vm_offset_t ptr); void efi_copy_finish(void); +void efi_copy_get_locations(uint64_t *dst, uint64_t *src, uint64_t *src_end); #endif /* _LOADER_EFI_COPY_H_ */