du -k VS ls -l (what I'm missing?)

[Matthew Seaman]

On Fri, Jul 02, 2004 at 01:04:32AM +0400, Alex K wrote:

> what do I miss here?
> sum of individual file sizes is much more than "total" in ls and more than du -k reports
> 
> bash-2.05b$ ls -l
> total 354112
> -rw-r--r--  1 lesha  wheel   98490960  1 ??? 12:29 88479E51B1D77190A2A8C882
> -rw-r--r--  1 lesha  wheel  299376716 25 ??? 15:20 F445555AA5CA2D90F33EE0F1
> -rw-r--r--  1 lesha  wheel  540729348  1 ??? 19:01 0C859D601337F1D26D68BA90
> -rw-r--r--  1 lesha  wheel  125204414 30 ??? 18:12 50922168AB8D4CB73FA39063
> -rw-r--r--  1 lesha  wheel  365164364  1 ??? 12:06 CBB789334BF480B9ED153EA8
> -rw-r--r--  1 lesha  wheel  209031053 30 ??? 19:05 B2AFAA6C8C68575BA97476F4
> -rw-r--r--  1 lesha  wheel  336457988 29 ??? 17:43 200DCA96AFFAF2FB08E3E279
> -rw-r--r--  1 lesha  wheel   40714776  1 ??? 18:16 6E30F671D9F305458A093617
> -rw-r--r--  1 lesha  wheel  209945132 25 ??? 15:29 A515D96BFAD85C294D4A9BB7
> -rw-r--r--  1 lesha  wheel  114632620  1 ??? 18:25 7868FE483F37D653109E67B3
> -rw-r--r--  1 lesha  wheel  242241614  1 ??? 19:02 75B7DC03642E00CE564C1FF6
> -rw-r--r--  1 lesha  wheel   42681134 25 ??? 15:29 F9C3246915327E44B9B0FD2C
> -rw-r--r--  1 lesha  wheel  165569384 30 ??? 18:54 4FCA6EC8E3AB33B33E3E5011
> bash-2.05b$ du -k
> 354114  .
> bash-2.05b$

The 'total' figure from ls(1) and the number shown by du(1) is the
total disk usage in blocks of 1024 bytes (if BLOCKSIZE=k is set in
your environment, which is the default) -- in this case, about a
factor of 7 smaller than the total of the file sizes.

Files can have 'holes' -- parts of the file that have never been
written to, although later parts of the file have.  Disk blocks are
not allocated for those unwritten areas.  If you use hexdump(1) on the
file, the holes will show up as a sequence of null bytes.

The way to tell if a file is holey is to compare the size of the file
against the number of blocks allocated for it using:

    % stat -f "%10z %6b %N" *

[ or

    % ls -ls *

where the 1st column is the number of blocks, the 6th is the filesize
in bytes]

If the filesize is significantly greater than the number of blocks
multiplied by the block size (stat(1) shows 512 byte blocks, ls(1)
shows 1024 byte blocks) then those files have holes in them.  It's
quite common to see this, for example, in files that are the backing
stores for databases.

Having holey files is not a problem, although some broken backup
software will tend to fill in all of the gaps with zeros, meaning that
occasionally you can't restore a file back onto the same partition it
was backed up from.  You can quite easily have a file whose apparent
size is larger than the partition holding it.

	Cheers,

	Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.                       26 The Paddocks
                                                      Savill Way
PGP: http://www.infracaninophile.co.uk/pgpkey         Marlow
Tel: +44 1628 476614                                  Bucks., SL7 1TH UK
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20040701/cce28b50/attachment.bin