vnode_pager_putpages errors and DOS?

[Uwe Doering]

Robert Watson wrote:
> On Fri, 8 Oct 2004, Steve Shorter wrote:
> 
>>>	I have some machines that run customers cgi stuff.
>>>These machines have started to hang and become unresponsive.
>>>At first I thought it was a hardware issue, but I discovered in 
>>>a cyclades log the following stuff that got logged to the
>>>console which explains the cause of the system hangs/failures.
>>>
>>>vnode_pager_putpages: residual I/O 65536 at 347
>>>vnode_pager_putpages: I/O error 28]
>>>vnode_pager_putpages: residual I/O 65536 at 285] 
>>
>>	Aha! also at the same time I get in syslog
>>
>>	/kernel: pid 6 (syncer), uid 0 on /chroot/tmp: file system full
>>
>>	Whats happening? Can a full filesystem bring the thing down? 
>>Ideas? Fixes? 
> 
> Ideally not, but many UNIX programs respond poorly to being out of memory
> and disk space ("No space, wot?").  Are you using a swap file, and if so,
> how did you create the swapfile?  Are you using sparse files much?

I wonder whether the unresponsiveness is actually just the result of the 
kernel spending most of the time in printf(), generating warning 
messages.  vnode_pager_generic_putpages() doesn't return any error in 
case of a write failure, so the caller (syncer in this case) isn't aware 
that the paging out failed, that is, it is supposed to carry on as if 
nothing happened.

So how about limiting the number of warnings to one per second?  UFS has 
similar code in order to curb "file system full" and the like.  Please 
consider trying the attached patch, which applies cleanly to 4-STABLE. 
It won't make the actual application causing these errors any happier, 
but it may eliminate the DoS aspect of the issue.

    Uwe
-- 
Uwe Doering         |  EscapeBox - Managed On-Demand UNIX Servers
gemini at geminix.org  |  http://www.escapebox.net
-------------- next part --------------
--- src/sys/vm/vnode_pager.c.orig	Fri Oct 31 11:39:38 2003
+++ src/sys/vm/vnode_pager.c	Sun Feb 15 02:38:21 2004
@@ -955,6 +955,7 @@
 	struct iovec aiov;
 	int error;
 	int ioflags;
+	static int last_elog, last_rlog;
 
 	object = vp->v_object;
 	count = bytecount / PAGE_SIZE;
@@ -1035,10 +1036,12 @@
 	cnt.v_vnodeout++;
 	cnt.v_vnodepgsout += ncount;
 
-	if (error) {
+	if (error && last_elog != time_second) {
+		last_elog = time_second;
 		printf("vnode_pager_putpages: I/O error %d\n", error);
 	}
-	if (auio.uio_resid) {
+	if (auio.uio_resid && last_rlog != time_second) {
+		last_rlog = time_second;
 		printf("vnode_pager_putpages: residual I/O %d at %lu\n",
 		    auio.uio_resid, (u_long)m[0]->pindex);
 	}