rmt as a bottleneck - Was: Weird behaviour of AIT-3 and (g)tar

[David Gilbert]

>>>>> "Bruce" == Bruce Evans <bde at zeta.org.au> writes:

>> Observe that I bypassed rmt; that bumped the transfer rate to
>> 10.976.153,96 Bytes/s, almost 30x faster. Should this really
>> happen? (And yes, I read rmt(8), but found nothing about this. :(
>> ).  Thanks for your help;

Bruce> ISTR that remote tars have a delay of 10 msec or so for each
Bruce> block because the protocol needs to talk after each block (it
Bruce> doesn't stream) and there is a TCP startup delay of this
Bruce> amount.

I have often considered the problem of remote tape devices.  By way of
introduction: I've done a lot of tape handling on UNIX and other
systems for complex block formats.  The UN*X tape model is rather
simplistic.

The primary problem with rmt (and UN*X tape in general) is that it
cannot stream.  If I signal "EOT" (end-of-tape) to tar, tar blithly
assumes I meant the block you just wrote caused an EOT.  Some tape
hardware approximates streaming by estimating EOT ahead of time --- it
certainly performs better, but it's not a great solution.

Some tape programs on UN*X attempt to buffer heavily in RAM (team,
etc)... but to overcome this shortcoming in the multi-tape case, they
all need multi-volume smarts.  Pretty much a mess.

So... yes... there's a chatter for every block because the protocol
needs to return the status of every block.

I keep threatening to write the difinitive tape suite for FreeBSD
... but something more important always comes along.  Problem is,
everything needs rejiging ... tar, dump, tape drivers, rmt, etc.

Dave.

-- 
============================================================================
|David Gilbert, Independent Contractor.       | Two things can only be     |
|Mail:       dave at daveg.ca                    |  equal if and only if they |
|http://daveg.ca                              |   are precisely opposite.  |
=========================================================GLO================