svn commit: r351522 - in head: sbin/ifconfig share/man/man4 sys/conf sys/kern sys/modules sys/modules/ktls_ocf sys/net sys/netinet sys/netinet/tcp_stacks sys/netinet6 sys/opencrypto sys/sys tools/t...

[John Baldwin]

On 8/27/19 10:05 AM, Peter Holm wrote:
> On Tue, Aug 27, 2019 at 09:02:31AM -0700, John Baldwin wrote:
>> On 8/27/19 7:39 AM, Peter Holm wrote:
>>> On Tue, Aug 27, 2019 at 12:01:57AM +0000, John Baldwin wrote:
>>>> Author: jhb
>>>> Date: Tue Aug 27 00:01:56 2019
>>>> New Revision: 351522
>>>> URL: https://svnweb.freebsd.org/changeset/base/351522
>>>>
>>>> Log:
>>>>   Add kernel-side support for in-kernel TLS.
>>>>   
>>>
>>> Could this be yours?
>>>
>>> 20190827 15:55:34 all (496/668): sendfile12.sh
>>> Aug 27 15:56:16 mercat1 kernel: pid 50036 (swap), jid 0, uid 0, was killed: out of swap space
>>> panic: non-ext_pgs mbuf with TLS session
>>
>> Possibly, though if sfio was freed and marked with 0xdeadc0de junk, then it
>> would trip over this assertion for any use-after-free.  I see in gdb that you
>> couldn't see sfio because of clang's poor debug info.  It would be really good
>> to try to find the contents of sfio to debug this further.
>>
>> You should be able to find it via 'bp->b_caller1' in frame 14:
>>
>> 'p *(struct sf_io *)bp->b_caller1'
>>
> 
> Here's a repeat where the involved files are compiled with "-O0":
> https://people.freebsd.org/~pho/stress/log/jhb009.txt

Ok, it looks like sfio->tls is just not being initialized to NULL in the
!KERN_TLS case and the malloc junk is leaking through (my fault):

(kgdb) p *(struct sf_io *)bp->b_caller1
$5 = {nios = 0x0, error = 0x0, npages = 0x1, so = 0xfffff808898d0000, m = 0xfffff808a3512200, tls = 0xdeadc0dedeadc0de, pa = 0xfffff804e6cdfc68}

Initially I thought about using M_ZERO, but we can just axe the 'tls'
member of 'sfio' entirely in the !KERN_TLS case since it's a private
structure.

Try this (untested) change):

Index: kern_sendfile.c
===================================================================
--- kern_sendfile.c	(revision 351522)
+++ kern_sendfile.c	(working copy)
@@ -88,7 +88,9 @@ struct sf_io {
 	int		npages;
 	struct socket	*so;
 	struct mbuf	*m;
+#ifdef KERN_TLS
 	struct ktls_session *tls;
+#endif
 	vm_page_t	pa[];
 };
 
@@ -266,7 +268,7 @@ sendfile_iodone(void *arg, vm_page_t *pg, int coun
 	if (!refcount_release(&sfio->nios))
 		return;
 
-#ifdef INVARIANTS
+#if defined(KERN_TLS) && defined(INVARIANTS)
 	if ((sfio->m->m_flags & M_EXT) != 0 &&
 	    sfio->m->m_ext.ext_type == EXT_PGS)
 		KASSERT(sfio->tls == sfio->m->m_ext.ext_pgs->tls,

-- 
John Baldwin