Network pipes

[John]

----- Diomidis Spinellis's Original Message -----
> > Luigi Rizzo wrote:
> > 
> > * the loss of encription on the channel is certainly something that might
> >   escape the attention of the user. I also wonder in how many cases you
> >   really need the extra performance to justify the extra plumbing
> >   mechanism.

Once you pass a certain order of magnitute, it becomes an overriding
priority. Thus the reason why many backup systems are hand crafted. 

> I felt the need for such functionality when moving GB data between
> different machines for creating a disk copy and backup to tape.  My
> requirements may be atypical, this is why I asked for input.

Your requirements are not atypical. There are folks out here dealing
with 100s & 1000s of TB.

> > * there are subtle implications of your new plumbing in the way
> >   processes are started. With "A | B | C" the shell first creates the
> >   pipes, then it can start the processes in any order, and they can
> >   individually fail to start without any direct consequence other
> >   than an I/O failure. "A |@ B |@ C" requires that you start things
> >   from the end of the chain (because you cannot start a process
> >   until you have a [socket] descriptor from the next stage in the
> >   chain), and if a process fails to start you cannot even start the
> >   next one in the sequence. Not that this is bad, just very different
> >   from regular pipes.
> 
> It is even worse.  You can not write "A |@ B |@ C" because sockets are
> created on the originating host.  For the above to work you would need a
> mechanism to create another socket between the B and C machines.  Maybe
> the syntax should be changed to make such constructions
> counterintuitive.

Syntactic consistency should be a high priority.

-john