svn commit: r326758 - in head/sys/i386: conf include

[Nathan Whitehorn]

On 12/12/17 11:32, John Baldwin wrote:
> On 12/11/17 5:26 AM, Eugene Grosbein wrote:
>> On 11.12.2017 16:19, Konstantin Belousov wrote:
>>> On Mon, Dec 11, 2017 at 04:32:37AM +0000, Conrad Meyer wrote:
>>>> Author: cem
>>>> Date: Mon Dec 11 04:32:37 2017
>>>> New Revision: 326758
>>>> URL: https://svnweb.freebsd.org/changeset/base/326758
>>>>
>>>> Log:
>>>>    i386: Bump KSTACK_PAGES default to match amd64
>>> i386 is not amd64, the change is wrong.
>>>
>>> i386 has the word size two times smaller than amd64, which makes typical
>>> frame smaller by 30-40% over same code on amd64. Also i386 has much
>>> smaller available KVA size (tens of MB) and KVA fragmentation is both
>>> more severe and more fatal due to this. I expect that your change will
>>> make any non-trivial load which creates enough threads to either fail
>>> randomly or deadlock.
>>>
>>> If somebody tries to fit large load onto i386 machine, he must know what to
>>> do and how to configure the kernel to adapt to the load (which does not
>>> require the recompilation).
>> Its very easy to get kernel stack overflow with 11-STABLE/i386
>> without any significant load due to abuse of kernel stack in many kernel subsystems
>> as shown in the https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219476
>>
>> Contrary, "enough threads" seems to be very non-trivial number of threads
>> and pretty unusual load pattern for i386 as I run several such systems
>> with kern.kstack_pages=4 for quite long time and have no problems.
>> No random fails, no deadlocks. And with 2 pages only 11-STABLE/i386 is just unusable
>> if one utilizes SCTP, IPv6, some WiFi connectivity, IPSEC or even very small ZFS pool.
>>
>> I still wonder if there is really such load pattern that creates "enough threads"
>> for i386 to make 4-pages stack troublesome.
> I suspect two things are at play in running out of stack in 10.x and later.
> 1) Usage of XSAVE for AVX, etc. state uses more of the kstack (as kib@ alludes
> to), and 2) clang.  Certainly for MIPS I have found that compiling with clang
> instead of gcc for mips64 gives a kernel that panics for stack overflow for any
> use of NFS.  It might be that this is due to something MIPS-specific, but it
> might be worthwhile retesting with kstack_pages=2 and building the kernel
> with CROSS_TOOLCHAIN=i386-gcc after installing the appropriate package.

For what it's worth, I see the same NFS-related stack overflows on 
mips64 with in-tree GCC 4.2.1.
-Nathan