kern/60367: Erroneous "(core dumped)" in process death message

[Valentin Nechayev]

>Number:         60367
>Category:       kern
>Synopsis:       Erroneous "(core dumped)" in process death message
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Dec 18 06:30:15 PST 2003
>Closed-Date:
>Last-Modified:
>Originator:     Valentin Nechayev
>Release:        FreeBSD 5.2-RC-20031208 i386
>Organization:
private
>Environment:
FreeBSD 5.2-RC
FreeBSD 4.8-RELEASE

>Description:

On mortal signal, sigexit() in sys/kern/kern_sig.c calls coredump()
for some signals (with SA_CORE flag in property table) and marks exit status
with WCOREFLAG if it returned 0.
coredump() in the same source file returns 0 on successful core write,
and errno codes in most cases, but not when core dump is prohibited
by resource limit (p->p_rlimit[RLIMIT_CORE].rlim_cur), instead returning 0.
I think it is simply typo. As result, kernel says "process exited on signal
N (core dumped)" when no core was even supposed to be dumped.

>How-To-Repeat:

ulimit -Sc 0 (in sh/bash) and run something which deads on SIGSEGV.

>Fix:

For 5.2:

--- kern_sig.c.0	Mon Nov  3 14:09:06 2003
+++ kern_sig.c	Thu Dec 18 16:20:00 2003
@@ -2546,7 +2546,7 @@
 	limit = p->p_rlimit[RLIMIT_CORE].rlim_cur;
 	if (limit == 0) {
 		PROC_UNLOCK(p);
-		return 0;
+		return (ENOSPC);
 	}
 	PROC_UNLOCK(p);
 
For 4.8, the same patch is to be apply in some different context
(without PROC_UNLOCK).

I selected ENOSPC, but really error code doesn't matter while it is !=0.
>Release-Note:
>Audit-Trail:
>Unformatted: