git: af949c590bd8 - main - Disable stack gap for ntpd during build.

Fri May 21 16:42:46 UTC 2021

Hi Cy,

pt., 21 maj 2021 o 16:46 Cy Schubert <Cy.Schubert at cschubert.com> napisał(a):
>
> In message <02078965-24BE-4F23-92D5-5E8E54A0C3E7 at freebsd.org>, Jessica
> Clarke w
> rites:
> > > On 21 May 2021, at 15:11, Marcin Wojtas <mw at semihalf.com> wrote:
> > >
> > > Hi Jess
> > >
> > > pt., 21 maj 2021 o 15:39 Jessica Clarke <jrtc27 at freebsd.org> napisał(a):
> > >>
> > >> On 21 May 2021, at 14:34, Marcin Wojtas <mw at FreeBSD.org> wrote:
> > >>>
> > >>> The branch main has been updated by mw:
> > >>>
> > >>> URL: https://cgit.FreeBSD.org/src/commit/?id=af949c590bd8a00a5973b5875d7e
> > 0fa6832ea64a
> > >>>
> > >>> commit af949c590bd8a00a5973b5875d7e0fa6832ea64a
> > >>> Author:     Marcin Wojtas <mw at FreeBSD.org>
> > >>> AuthorDate: 2021-05-21 09:29:22 +0000
> > >>> Commit:     Marcin Wojtas <mw at FreeBSD.org>
> > >>> CommitDate: 2021-05-21 13:33:06 +0000
> > >>>
> > >>>   Disable stack gap for ntpd during build.
> > >>>
> > >>>   When starting, ntpd calls setrlimit(2) to limit maximum size of its
> > >>>   stack. The stack limit chosen by ntpd is 200K, so when stack gap
> > >>>   is enabled, the stack gap is larger than this limit, which results
> > >>>   in ntpd crashing.
> > >>
> > >> Isn’t the bug that the unusable gap counts as usage?
> > >>
> > >> Jess
> > >>
> > >
> > > An alternative solution was submitted
> > > (https://reviews.freebsd.org/D29832), so that to extend the limit for
> > > ntpd, but eventually it was recommended to simple disable the stack
> > > gap for it until it's fixed upstream (see the last comment in the
> > > linked revision).
> >
> > That’s my point, there is nothing to “fix” upstream. NTPD uses less tha
> > n 200K
> > of stack, thus it is perfectly reasonable for it to set its limit to that. Th
> > e
> > fact that FreeBSD decides to count an arbitrary, non-deterministic amount of
> > additional unusable virtual address space towards that limit is not its fault
> > ,
> > but a bug in FreeBSD that needs to be fixed as it’s entirely unreasonable f
> > or
> > applications to have to account for that.
>
> This latest problem is not stack gap. It is PIE.
>

I have to disagree.
ntpd does not start because of stack gap, not PIE, even though it may
seem like PIE causes this. This is due to the fact that stack gap is
disabled if PIE is disabled. Because of that value of sysctl
kern.elf64.aslr.stack_gap does not matter when kern.elf64.aslr.pie_enable
is set to 0. When pie_enabled is set to 1 and stack gap is enabled, then
ntpd fails to start, but when pie_enabled is set to 1 and stack_gap
is set to 0, then ntpd starts without any issue. We verified this on
FreeBSD-CURRENT snapshot from 2021-05-20.

The fact that this is a stack gap issue can be verified using following
procedure:
1. Install FreeBSD-CURRENT snapshot from 2021-05-20 using default
configuration.
2. On a newly installed system start ntpd. With default configuration
it should start successfully.
3. Set sysctl kern.elf64.aslr.pie_enable=1 and start ntpd. This time ntpd
should fail. An entry indicating that ntpd was killed because of signal
11 should be visible in /var/log/messages.
4. Set sysctl kern.elf64.aslr.stack_gap=0 and start ntpd once again. This
time ntpd should start even though pie_enable is set to 1.

Exact log from the boot it was tested:
root at freebsd-ntpd-test:~ # sysctl -a | grep aslr
kern.elf32.aslr.stack_gap: 3
kern.elf32.aslr.honor_sbrk: 1
kern.elf32.aslr.pie_enable: 0
kern.elf32.aslr.enable: 0
kern.elf64.aslr.stack_gap: 3
kern.elf64.aslr.honor_sbrk: 1
kern.elf64.aslr.pie_enable: 0
kern.elf64.aslr.enable: 0
vm.aslr_restarts: 0
root at freebsd-ntpd-test:~ # ntpd
root at freebsd-ntpd-test:~ # ps aux | grep ntpd
root  826   0.0  0.2 22060 6960  -  Ss   17:38    0:00.01 ntpd
root  828   0.0  0.1 12976 2416  0  S+   17:38    0:00.00 grep ntpd
root at freebsd-ntpd-test:~ # killall ntpd
root at freebsd-ntpd-test:~ # ps aux | grep ntpd
root  831   0.0  0.1 12976 2416  0  S+   17:38    0:00.00 grep ntpd
root at freebsd-ntpd-test:~ # sysctl kern.elf64.aslr.pie_enable=1
kern.elf64.aslr.pie_enable: 0 -> 1
root at freebsd-ntpd-test:~ # ntpd
root at freebsd-ntpd-test:~ # ps aux | grep ntpd
root  836   0.0  0.1 14128 2452  0  S+   17:39    0:00.00 grep ntpd
root at freebsd-ntpd-test:~ # cat /var/log/messages | tail
May 21 17:38:25 freebsd-ntpd-test ntpd[826]: ntpd exiting on signal 15
(Terminated)
May 21 17:39:14 freebsd-ntpd-test ntpd[833]: ntpd 4.2.8p15-a (1): Starting
May 21 17:39:14 freebsd-ntpd-test ntpd[833]: Command line: ntpd
May 21 17:39:14 freebsd-ntpd-test ntpd[833]:
----------------------------------------------------
May 21 17:39:14 freebsd-ntpd-test ntpd[833]: ntp-4 is maintained by
Network Time Foundation,
May 21 17:39:14 freebsd-ntpd-test ntpd[833]: Inc. (NTF), a non-profit
501(c)(3) public-benefit
May 21 17:39:14 freebsd-ntpd-test ntpd[833]: corporation.  Support and
training for ntp-4 are
May 21 17:39:14 freebsd-ntpd-test ntpd[833]: available at
https://www.nwtime.org/support
May 21 17:39:14 freebsd-ntpd-test ntpd[833]:
----------------------------------------------------
May 21 17:39:14 freebsd-ntpd-test kernel: pid 834 (ntpd), jid 0, uid
0: exited on signal 11 (core dumped)
root at freebsd-ntpd-test:~ # sysctl kern.elf64.aslr.stack_gap=0
kern.elf64.aslr.stack_gap: 3 -> 0
root at freebsd-ntpd-test:~ # sysctl -a | grep aslr
kern.elf32.aslr.stack_gap: 3
kern.elf32.aslr.honor_sbrk: 1
kern.elf32.aslr.pie_enable: 0
kern.elf32.aslr.enable: 0
kern.elf64.aslr.stack_gap: 0
kern.elf64.aslr.honor_sbrk: 1
kern.elf64.aslr.pie_enable: 1
kern.elf64.aslr.enable: 0
vm.aslr_restarts: 1
root at freebsd-ntpd-test:~ # ntpd
root at freebsd-ntpd-test:~ # ps aux | grep ntpd
root  845   0.0  0.2 22060 6924  -  Ss   17:40    0:00.01 ntpd
root  847   0.0  0.1 12976 2440  0  S+   17:40    0:00.00 grep ntpd
root at freebsd-ntpd-test:~ # cat /var/log/messages | tail
May 21 17:39:14 freebsd-ntpd-test kernel: pid 834 (ntpd), jid 0, uid
0: exited on signal 11 (core dumped)
May 21 17:40:52 freebsd-ntpd-test ntpd[844]: ntpd 4.2.8p15-a (1): Starting
May 21 17:40:52 freebsd-ntpd-test ntpd[844]: Command line: ntpd
May 21 17:40:52 freebsd-ntpd-test ntpd[844]:
----------------------------------------------------
May 21 17:40:52 freebsd-ntpd-test ntpd[844]: ntp-4 is maintained by
Network Time Foundation,
May 21 17:40:52 freebsd-ntpd-test ntpd[844]: Inc. (NTF), a non-profit
501(c)(3) public-benefit
May 21 17:40:52 freebsd-ntpd-test ntpd[844]: corporation.  Support and
training for ntp-4 are
May 21 17:40:52 freebsd-ntpd-test ntpd[844]: available at
https://www.nwtime.org/support
May 21 17:40:52 freebsd-ntpd-test ntpd[844]:
----------------------------------------------------
May 21 17:40:52 freebsd-ntpd-test ntpd[845]: leapsecond file
('/var/db/ntpd.leap-seconds.list'): stat failed: No such file or
directory
root at freebsd-ntpd-test:~ # killall ntpd

Best regards,
Marcin