nagios and freebsd threads issue : help please ...

Christophe Yayon lists at
Wed Aug 24 05:22:58 GMT 2005

Hi all,

Here is my copy/paste from freebsd-hackers to nagios-devel list, and the 
answer from a Nagios developper.

Christophe Yayon wrote:

 >> Hi again,
 >> After some discussions on freebsd-hackers mailling list, here is a 
resume :
 >> 1. There a recommendation (or a suggestion) for what to do after a 
fork() :
 >> "In other words "It is suggested that programs that use fork() call an
 >> exec function very soon afterwards in the child process, thus resetting
 >> all states. In the meantime, only a short list of async-signal-safe
 >> library routines are promised to be available."
 >> Note *suggested*. This is a recommendation to protect against a shoddy
 >> pthread-implementation. The thread specifications rule that only the
 >> thread calling fork() is duplicated, which initially leads to the
 >> recommendation (other threads holding locks aren't around to release 
 >> in the new execution context).
 >> 2. it appears that Nagios do after a fork :
 >> in base/util.c:
 >>         (1) Become the process group leader by calling setpgid(0, 0);
 >>         (2) something called set_all_macro_environemt_vars(TRUE).
 >>             This calls snprintf a bunch, as well as set variables
 >>             by saving them to malloced memory.  This save is done
 >>             with strcpy and strcat.  setenv is then called to try to
 >>             export them.  memory is then freed with free(3).
 >>         (3) All signal handlers are reset
 >>         (4) The right part of the pipe is closed
 >>         (5) sigalarm handler is created and an alarm set.
 >>         (6) Checks to see if it executing an embedded perl script,
 >>             then tries to execute it if so.  This has the feel of
 >>             being too much after the fork.
 >>         (7) Calls popen on the command if not.
 >>         (8) Reads the output of the command using fgets.
 >>         (9) closes the other end of the pipe
 >>         (10) unsets all env vars.
 >>         (11) Calls _exit()
 >> in base/checks.c
 >>         (1) set_all_macro_environment_vars(TRUE)
 >>         (2) forks again
 >>         (3) granchild:
 >>                 resets handler, setpgid, etc.
 >>                 if perl script, do embedded perl, otherwise popen.
 >>                 lots of read/write to pipe.
 >> likewise in base/commands.c fork is also called for similar things.
 >> There's other places that also call popen...
 >> 3. You can only execute async-signal-safe functions after a fork()
 >> from a threaded application.  free(), malloc(), popen(), fgets(),
 >> are not async-signal-safe.

In a proper implementation they are. Read malloc/malloc.c from
glibc-2.3.5 and you'll see. The first line of it reads

"/* Malloc implementation for multiple threads without lock contention"

fgets() must also be async-safe, since it's passed its storage-buffer
from the calling function. It can contain races if several threads (or
programs for that matter) tries to read FIFO's at the same time or are
trying to store things to the same piece of memory, but that's neither
new, strange or in any way non-obvious. Obviously, fgets() relies on
lower-level IO code which must be thread-safe (read() in this case) on
account of them being syscalls inside multitasking kernels.

popen() forks and calls execve immediately. If this isn't thread-safe
then there's no way of executing external programs in multithreaded
applications short of implementing popen() directly (which isn't exactly
difficult, but still).

 >>  The list of async-signal-safe functions
 >> are here:
 >> The restriction on fork() is here (20th bullet down):

Both of those links point to the same document, which is just the
frameset for the navigation-frames.

For async-safe functions, this is the proper url;

For the fork() specification, the doc is here;

The 20'th bullet is this;
"A process shall be created with a single thread. If a multi-threaded
process calls fork(), the new process shall contain a replica of the
calling thread and its entire address space, possibly including the
states of mutexes and other resources. Consequently, to avoid errors,
the child process may only execute async-signal-safe operations until
such time as one of the exec functions is called. [THR] [Option Start]
Fork handlers may be established by means of the pthread_atfork()
function in order to maintain application invariants across fork()
calls. [Option End]

When the application calls fork() from a signal handler and any of the
fork handlers registered by pthread_atfork() calls a function that is
not asynch-signal-safe, the behavior is undefined."

Also note that "From the application's perspective, a fork() call should
appear atomic." which implicitly states fork() as an async-safe function
although the following execution may not be. It also warns that improper
implementations makes it less so.

 >> 4. Some FreeBSD developpers think that handling fork() in libpthread 
 >> probably libthr) than was done in libc_r.  We thought it better not 
to try
 >> and reinitialize libpthread (and to some extent libc) because
 >> it is messy and to expose non-portable applications.

This is funny, because nagios apparently runs properly on Linux, HPUX,
Solaris, Irix, AIX and Tru64. To me that seems to indicate that Nagios
is very portable indeed and that the BSD fellows somehow botched it. I
might be wrong, but...

 >> Possibles solutions :
 >> a. (the best, i think) Trying to modify Nagios code to respect the
 >> recommendation (1.). We are talking about portability and not
 >> performance...

This would involve a fairly large change in the way things are done. I
for one am all for implementing a different parallelisation mechanism
but I'm fairly certain Ethan won't be too thrilled if I rewrite 40% of
the code that's currently the Nagios core.

 >> b. a possible workaround for Nagios FreeBSD (and i think other Unix
 >> systems, except Linux) is to use another threads library. For FreeBSD it
 >> seems that uising GNU/pth (which is in the ports) seems to completely
 >> resolve the problem (but i think it's ugly to have to use another -not
 >> native- threads lib...).
 >> What do you think about this ?

In summary; Some thread-libraries work while others don't (the native
*BSD one being the only one that doesn't), I'd say it's time to fix that
thread-library, although I favor the rewrite-nagios approach as an
exercise in intellectual masturbation and would be quite willing to do
the actual work of it, provided I can be somewhat sure it isn't wasted.

 >> Sorry for my english (i am french...)

Your english is far better than most native english speakers I've come

 >> PS : thanks to all freebsd-hackers posters which permit to resume the
 >> problem (Warner Losh, Daniel Eischen, Alexey Vesnin).

More information about the freebsd-hackers mailing list