Large discrepancy in reported disk usage on USR partition

[Jeremy Chadwick]

On Fri, Oct 31, 2008 at 11:50:39AM +1030, Brendan Hart wrote:
> >> #: df -h
> >> Filesystem          Size    Used   Avail Capacity  Mounted on
> >> /dev/aacd0s1a       496M    163M     293M    36%    /
> >> devfs               1.0K    1.0K     0B      100%   /dev
> >> /dev/aacd0s1e       496M    15M      441M    3%     /tmp
> >> /dev/aacd0s1f        28G    25G      1.2G    96%    /usr
> >> /dev/aacd0s1d       1.9G    429M     1.3G    24%    /var
> 
> > Is this output untruncated? Is df really df or an alias to 'df -t nonfs'?
> 
> Yes, it really is the untruncated output of "df -h". I also tried the "df -t
> nonfs" and it gives exactly the same output as "df". What are you expecting
> that is not present in the output ?
> 
> > Is it possible that nfs directory got written to /usr at some point in
> time? 
> > You would only notice this with du if the nfs directory is unmounted.
> > Unmount it and ls -al /usr/mountpoint should only give you an empty dir
> 
> Bingo!! That is exactly the problem. An NFS mount was hiding a 17G local dir
> which had an old copy of the entire NFS mounted dir. I guess it must have
> been written incorrectly to this standby server by RSYNC before the NFS
> mount was put in place. I will add an exclusion to rsync to make sure it
> does not happen again even if the NFS dir is not mounted.
> 
> Thank you for your help, you have saved me much time rebuilding this server.

Can either of you outline what exactly happened here?  I'm trying to
figure out how an "NFS mount was hiding a 17G local dir", when there's
no NFS mounts shown in the above df output.  This is purely an ignorant
question on my part, but I'm not able to piece together what happened.

-- 
| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |