multi volume dump with gzip/crypt - solved
rwatson at FreeBSD.org
Sun Mar 2 11:44:02 UTC 2008
On Sat, 1 Mar 2008, Martin Laabs wrote:
> I solved the dump problem in a not very clean way. First I changed dumps
> behaviour to ignore the SIGPIPE signal when the -P and -a option is given.
> The problem with occured than was the loss of data because of the race
> condition between "write date to pipe" and "receive SIGPIPE".
> I solved that problem with a script that takes the data from the pipe of
> dump, compress it and sends a dedicated SIGPIPE to the three dump processes
> when the output volume is nealy full. After that it flushs the rest out of
> the pipe to avoid data loss.
SIGPIPE occurs when a sending process sends data that the recipient doesn't
expect, as the recipient has (presumably) closed the socket because it doesn't
expect further data. Stream sockets are intended to provided buffered,
reliable communication only when open, so generally protocols running over
stream sockets (be it TCP, UNIX domain sockets, etc) have an explicit protocol
close sequence so that the two endpoints reach mutual agreement that it's time
to close the sockets and avoid data loss associated with unexpected assymetric
close. If the recipient closed the socket while the sender is still writing
to the socket, you may "lose" up to the size of the receive socket buffer,
which from the perspective of the sender may have been reported as
successfully delivered. The receive socket buffer, FYI, is effectively the
amount of data that the recipient has agreed can be locally buffered before
causing the sender to block.
Robert N M Watson
University of Cambridge
More information about the freebsd-hackers