streaming tape problem ...

[Gunther Schadow]

Hi,

does anyone of you know of some smart buffering dd-like program
that can use asynchronous I/O (or some other smart trick) to
maintain a constant bit-rate writing to a SCSI tape? I have a
TK88 DLT IV on my (one and only) SCSI bus now and I can't make
dump(8) or even something like "dd if=/dev/rda0f" maintain
enough output to the tape for the highest density option. The
only way I can stream backups now is using dump with -b 128
after turning off the compression option on the drive.

My question is whether dump(8), tar(1) and perhaps dd(1) could
be modified to use asynchronous IO and then maintain a buffer
just as the typical CDROM writers do it to prevent buffer underrun.
If that is not feasible, what would a program such as dd have
to do to maintain a constant output rate? I am thinking that
if dump and tar can't be modified to use async IO, then may
be one could modify dd and use it as a smart buffer like this:

dump -au0 -b 128 -f - /usr \
|dd of=/dev/nsa0 bs=128k sb=1280k

where the sb option would say that a 1280k buffer were to be
used, filled to the highwater mark before output would begin.
Then output would be done using aio_write such that the upstream
programs such as dump would not be blocked unless the buffer
were full.

On my VAX6460 with the TU81 9-track tape drive I did a similar
trick like this:

tar cvb 20 -f - /usr5/gcc-2.7.2 \
|dd ibs=2048000 of=/dev/nrmt0h obs=10240

that seemed like it would help. The idea here was that the large
ibs would get filled before dd starts outputting smaller
pieces. On the multi CPU machine with the right OS and fingers
crossed one could expect that the pipe would not be blocked
so much (some OS buffering in the pipe.) But still it may be
more wishful thinking. I'm afraid that the dd still blocks the
upstream pipe as long as the buffer is written to tape and then
ends up with an empty buffer.

I just found another interesting article:

http://www.dmst.aueb.gr/dds/pubs/trade/1999-login-dd/html/dd.html

this auther suggests to put an additional dd process into the pipe
for buffering:

tar cvb 20 -f - /usr5/gcc-2.7.2 \
|dd obs=512 \
|dd obs=512 of=/dev/nrmt0h

I will try that as soon as my backup finishes. It is streaming now,
just not in compressed mode.

regards
-Gunther

-- 
Gunther Schadow, M.D., Ph.D.                    gschadow at regenstrief.org
Medical Information Scientist      Regenstrief Institute for Health Care
Adjunct Assistant Professor        Indiana University School of Medicine
tel:1(317)630-7960                         http://aurora.regenstrief.org