multi volume dump with gzip/crypt - solved

[Robert Watson]

On Sat, 1 Mar 2008, Martin Laabs wrote:

> I solved the dump problem in a not very clean way. First I changed dumps 
> behaviour to ignore the SIGPIPE signal when the -P and -a option is given. 
> The problem with occured than was the loss of data because of the race 
> condition between "write date to pipe" and "receive SIGPIPE".
>
> I solved that problem with a script that takes the data from the pipe of 
> dump, compress it and sends a dedicated SIGPIPE to the three dump processes 
> when the output volume is nealy full. After that it flushs the rest out of 
> the pipe to avoid data loss.

SIGPIPE occurs when a sending process sends data that the recipient doesn't 
expect, as the recipient has (presumably) closed the socket because it doesn't 
expect further data.  Stream sockets are intended to provided buffered, 
reliable communication only when open, so generally protocols running over 
stream sockets (be it TCP, UNIX domain sockets, etc) have an explicit protocol 
close sequence so that the two endpoints reach mutual agreement that it's time 
to close the sockets and avoid data loss associated with unexpected assymetric 
close.  If the recipient closed the socket while the sender is still writing 
to the socket, you may "lose" up to the size of the receive socket buffer, 
which from the perspective of the sender may have been reported as 
successfully delivered.  The receive socket buffer, FYI, is effectively the 
amount of data that the recipient has agreed can be locally buffered before 
causing the sender to block.

Robert N M Watson
Computer Laboratory
University of Cambridge