sendfile data corruption

[Robert Watson]

On Sun, 12 Nov 2006, Fabian Keil wrote:

> Fabian Keil <freebsd-listen at fabiankeil.de> wrote:
>
>> Andre Oppermann <andre at freebsd.org> wrote:
>
>>> Please try this patch:
>>>
>>>   http://people.freebsd.org/~andre/sendfile_fix-20061112.diff
>>>
>>> It fixes apache 2.0.59 for me.
>>
>> For me too, but I'm still seeing problems with Gatling/0.8.
>
> I just had to reboot the system and noticed several LORs before
> the login prompt. Because of this one:
>
> lock order reversal:
> 1st 0xc070b6a8 Giant (sleep mutex) @ /usr/src/sys/kern/uipc_syscalls.c:1335
> 2nd 0xc27e5b10 inp (udpinp) @ /usr/src/sys/netinet/udp_usrreq.c:1120
> 3rd 0xc27e30e0 so_snd (sleep mutex) @ /usr/src/sys/kern/uipc_sockbuf.c:95

This is the correct lock order, so there must be a new lock order occuring 
earlier that gets added as the first order causing this one to appear as the 
incorrect order.  The following patch may help in making the right lock order 
problem come to surface:

Index: subr_witness.c
===================================================================
RCS file: /home/ncvs/src/sys/kern/subr_witness.c,v
retrieving revision 1.220
diff -u -r1.220 subr_witness.c
--- subr_witness.c	11 Nov 2006 03:18:06 -0000	1.220
+++ subr_witness.c	13 Nov 2006 11:14:15 -0000
@@ -325,6 +325,7 @@
  	/*
  	 * UDP/IP
  	 */
+	{ "Giant", &lock_class_mtx_sleep },
  	{ "udp", &lock_class_mtx_sleep },
  	{ "udpinp", &lock_class_mtx_sleep },
  	{ "so_snd", &lock_class_mtx_sleep },

> KDB: stack backtrace:
> db_trace_self_wrapper(c069ae0c) at db_trace_self_wrapper+0x25
> kdb_backtrace(0,0,c071a380,c071a060,c06d24c4,...) at kdb_backtrace+0x29
> witness_checkorder(c27e30e0,9,c069fc63,5f) at witness_checkorder+0x586
> _mtx_lock_flags(c27e30e0,0,c069fc63,5f,c27e5b10,...) at _mtx_lock_flags+0x84
> socantsendmore(c27e3000,c27e5b10,0,c06a78e3,460,...) at socantsendmore+0x1d
> udp_shutdown(c27e3000,0,d4b01d04,c2662700,d4b01c84,...) at udp_shutdown+0x3a
> soshutdown(c27e3000,2,c2776af8,0,c2662700,...) at soshutdown+0x37
> shutdown(c2662700,d4b01d04) at shutdown+0x5d
> syscall(b0003b,822003b,bfbf003b,bfbfc5d0,bfbfc6f0,...) at syscall+0x256
> Xint0x80_syscall() at Xint0x80_syscall+0x1f
> --- syscall (134, FreeBSD ELF32, shutdown), eip = 0x2810a277, esp = 0xbfbfc5bc, ebp = 0xbfbfc608 ---
>
> I assume there is a connection to your patch.
>
> For the rest of them please have a look at:
> http://www.fabiankeil.de/tmp/freebsd/dmesg-with-lors.txt
>
> Fabian
> -- 
> http://www.fabiankeil.de/
>