git: 19e008e7f79c - main - crt_malloc: Be more persistent when handling mmap() failure

From: Konstantin Belousov <kib_at_FreeBSD.org>
Date: Tue, 12 Oct 2021 23:45:41 UTC
The branch main has been updated by kib:

URL: https://cgit.FreeBSD.org/src/commit/?id=19e008e7f79ce55182d227be8513b3fa520471d8

commit 19e008e7f79ce55182d227be8513b3fa520471d8
Author:     Konstantin Belousov <kib@FreeBSD.org>
AuthorDate: 2021-10-11 16:13:31 +0000
Commit:     Konstantin Belousov <kib@FreeBSD.org>
CommitDate: 2021-10-12 23:37:09 +0000

    crt_malloc: Be more persistent when handling mmap() failure
    
    In the situation with limited address space, together with
    fragmentation, it is possible for mmap() request in morecore() to fail
    when asking for required size + NPOOLPAGES, but succeed without the
    addend.  Retry allocation there.
    
    PR:     259076
    Reported by:    Denis Koreshkov <dynamic-wind@mail.ru>
    Reviewed by:    arichardson
    Sponsored by:   The FreeBSD Foundation
    MFC after:      1 week
    Differential revision:  https://reviews.freebsd.org/D32474
---
 libexec/rtld-elf/rtld_malloc.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/libexec/rtld-elf/rtld_malloc.c b/libexec/rtld-elf/rtld_malloc.c
index f22c3c727c73..64218b5bb786 100644
--- a/libexec/rtld-elf/rtld_malloc.c
+++ b/libexec/rtld-elf/rtld_malloc.c
@@ -184,7 +184,9 @@ morecore(int bucket)
 		nblks = 1;
 	}
 	if (amt > pagepool_end - pagepool_start)
-		if (morepages(amt/pagesz + NPOOLPAGES) == 0)
+		if (morepages(amt / pagesz + NPOOLPAGES) == 0 &&
+		    /* Retry with min required size */
+		    morepages(amt / pagesz) == 0)
 			return;
 	op = (union overhead *)pagepool_start;
 	pagepool_start += amt;
@@ -269,6 +271,8 @@ morepages(int n)
 		}
 	}
 
+	if (pagepool_start == MAP_FAILED)
+		pagepool_start = 0;
 	offset = (uintptr_t)pagepool_start - rounddown2(
 	    (uintptr_t)pagepool_start, pagesz);