"tar tfv /dev/cd0" speedup patch

[Duane H. Hesser]

Tim and Juergen

I have a couple of suggestions which may help you with what you are
trying to do.

First, though, I can confirm that Unix character special tape
drivers since at least V7 have always ignored seeks.  They happily
return the requested offset without feeling the need to actually
*do* anything.  The FreeBSD scsi_sa driver continues this tradition.

Historically, all other Unix character special storage devices have
been unable to respond to  SEEK_END, presumably because they are
either removable medium devices or hard disks which have movable
"partitions", and no one ever bothered to arrange to query the
medium for size.  These devices do support seeking, and in fact it
is readily possibly to seek past the end of the file, just as for
regular files.

Unix block special tape drivers have historically supported seeking.
Somebody decided that FreeBSD doesn't need block devices and tossed
them out;  I think that's unfortunate.  Last time I looked (long time
ago) Linux had block device tapes, which did seek, and character 
special tape devices which did not.  (Correct me if I'm wrong, please).

I have a tar-compatible package of my own (tartools) which is about 20
years old but features the ability to fast_skip file data when listing or
skipping unselected files on extract.  It uses either seeking or mtio,
and handles the TAPE vs non-TAPE character special problem by simply
checking to see if the device responds to mtio commands.  If it does, it's
a tape, and can't seek (but can zip along using mtio).  

The last time I compiled it on Linux, the same situation applied, but
that was about 8 years ago, and according to Jung-uk Kim the linux folks
have finally done something about it.  The 32 bit version of the ioctl
does appear in the linux emulator in

/usr/src/sys/compat/linux/linux_ioctl.c

(I'm on freebsd 6.4/i386), and is implemented using an "fo_ioctl()"
with argument DIOCGMEDIASIZE, which /usr/include/sys/disk.h defines
as "_IOR('d', 129, u_int)".  That appears to be a valid ioct arg,
because several freebsd programs, including bsdlabel.c use 

ioctl(f, DIOCGMEDIASIZE, &mediasize)

where mediasize is declared as off_t, which seems wrong on recent 
freebsds, where off_t is 8 bytes, but u_int 

In any case, an ioctl isn't going to be very portable at this point
(although very handy).   There's another way to get medium size if
you need it.  Tartools use medium size to enable multi-volume
archives, and until recently required user input by option or "rc"
file to specify volume size. I've been doing extensive upgrades and
bug fixes on tartools the last 3 months, and I've recently added
code which will now get the size of USB sticks, compact flash, etc.
(even floppies) without requiring a command line argument.

It is possible to seek way past the end of a character special file
(and never know it).  This isn't a problem for single file archives
(although it might be risky) but if you want multiple volumes you
really need a size (which may be less than the medium size).

You can seek past the end of the medium, but you'll get an error if
you try to read or write there, so it is possible to get an accurate
size using a binary "seek and read" strategy to discover the highest
sector number which is readable).   

This is getting a little long, so let me summarize the suggestions.

1.  Disambiguate character special tape devices from other character
    special devices by issuing an mtio "status" ioctl (MTIOCGET)
	and checking the 'mt_type' byte in the status structure.  If
	it is 0, the device is not a tape and will support seeking
	(presuming it is a storage device).  It the type is non-zero
	then you can mark it non-seekable and either quit there or
	program the code to do the skipping via mtio.

2.  If you need the size of a device partition or removable
    medium, use a binary search algorithm which seeks forward
	until a read at the offset fails, then cutting the offset
	in half, seeking and reading again, etc.  It is usually
	possible to converge on a sector withing a couple of dozen
    seeks.




On Sat, 20 Feb 2010 01:00:40 -0500
Jung-uk Kim <jkim at freebsd.org> wrote:

> On Saturday 20 February 2010 12:20 am, Tim Kientzle wrote:
> > Juergen,
> >
> > I was looking at your Linux code here and thought
> > the technique of trying lseek(SEEK_END) might work.
> > Unfortunately, it doesn't: lseek(fd, 0, SEEK_END) gives
> > zero for both /dev/sa0 (a tape drive) and /dev/cd0
> > (an optical drive).  Are you sure it works on Linux?
> 
> Can you please try ioctl(fd, BLKGETSIZE64, &some_uint64_var) or 
> ioctl(fd, BLKGETSIZE, &some_u_long_var)?
> 
> Jung-uk Kim
> _______________________________________________
> freebsd-hackers at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org"
> 
---------
Duane.Hesser at gmail.com