svn commit: r326758 - in head/sys/i386: conf include

[Konstantin Belousov]

On Mon, Dec 11, 2017 at 07:33:08AM -0800, Rodney W. Grimes wrote:
> > On Mon, Dec 11, 2017 at 07:09:10AM -0800, Rodney W. Grimes wrote:
> > > The current comment about a pcb, I thought that code was changed
> > > so we only put the pointer to a pcb on the stack.
> > 
> > pcb is on top of the stack, followed by the userspace FPU registers save
> > area.  I do not see any sense in existence of pcb in modern kernel, it is
> > a remnant of the user area that was swappable.  Currently pcb is swappable
> > as well, but the value of this is much less then the overhead we pay by
> > the stack space reduction.
> 
> How about KSTACK_PAGES should be KSTAKE_BYTES/PAGESIZE, and we
> need a much better formula for define of KSTACK_BYTES so that
> these facts are more accurately defined?
The FPU save area size depends on CPU we booted on, so if we increase
(or decrease) it statically, we make a decision that is not good for
one or another generation of processors.

> 
> And a compile time assert if this ever growes to something unreasonable
> that would cause other issues.
So far it is an overkill, Intel did not passed the page boundary with all
extensions defined.

> 
> I fully agree with you that just bumping KSTACK_PAGES is very much
> the wrong way to fix the i386 issue of certain code not running,
> it is that code that should be examined for over usage of the stack.
In fact there is a tunable which I added to avoid recompiling kernel if
specific load requires large stack, keeping ZFS in mind.

> 
> I can assert that the base i386 system is very usable for tons of
> things without this change, I have at least 30 VM's running FreeBSD
> 11.1/i386 in some very small footprints, typically 64MB, that have
> zero issues.  But then they are not using any of the code that
> sited as problem areas.
> 
> > FPU save area is the on of the problem which makes us increase the amd64
> > stack size, AVX or even AVX512 make the things much worse.  It is unlikely
> > that somebody would run 32bit kernel on machine capable of that extensions,
> > i.e. Haswell or Skylake.
> 
> Your igonoring the virutalization world, host is a skylake or haswell, guest
> is i386 as it has small memory needs and no use to waste half of all pointers.
If you are serious about avoiding the waste, then you would also disable
unused extensions eating save area space, e.g. AVX512.  AVX might be still
useful (but not required) if AESNI or SHA extensions are used.

> 
> We need to break the developers model that i386 is dead and that i386 is
> not running on extremly modern hardware due to the factor of virtualization.
> 
> Output from one of my VM's running inside bhyve:
> 
>  # uname -a
> FreeBSD filestore.dnsmgr.net 11.1-RELEASE FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 04:10:47 UTC 2017     root at releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  i386
> 
>  # dmesg | head -24
> Copyright (c) 1992-2017 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>         The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 02:08:28 UTC 2017
>     root at releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
> FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on LLVM 4.0.0)
> VT(vga): resolution 640x480
> info: [drm] Initialized drm 1.1.0 20060810
> CPU: Intel(R) Core(TM) i5-3210M CPU @ 2.50GHz (2494.39-MHz K8-class CPU)
This is quite old processor, IvyBridge is 6 generations old :).