kern/67706: cvs update over a union mount slows and then slowly hangs the system

[JP Dinger]

>Number:         67706
>Category:       kern
>Synopsis:       cvs update over a union mount slows and then slowly hangs the system
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Jun 08 07:30:27 GMT 2004
>Closed-Date:
>Last-Modified:
>Originator:     JP Dinger
>Release:        4.9-REL-p9, 4.9-REL-p4, 4.10-REL
>Organization:
>Environment:
FreeBSD cazorp.gate5.de 4.9-RELEASE-p9 FreeBSD 4.9-RELEASE-p9 #1: Fri Jun  4 14:08:43 CEST 2004     root at cazorp.gate5.de:/usr/obj/usr/src/sys/PROZACII  i386

Also verified with GENERIC, of both 4.9-p4 and 4.10 (switched back from 4.10 to veryfy it wasn't a regression).
>Description:
With /usr/ports union mounted on /data/ports, cd /usr;cvs checkout ports 
will work but cvs update will make the machine unbearably slow, until no 
commands will work anymore. Symptoms include increasing slowness until
total halt, but switching vtys will keep working. Disc and network activity
will drop to zero as well, until interrupts/sec matches HZ. vmstat and top
(when they have been running before, starting them won't be doable) will
continue to update, however. The system will be 99.x% idle and run
0.(9-x)% sys, but no intr or user. Swap is not touched.
Namei and related cache stats are empty.

>How-To-Repeat:
df -h /data /usr
Filesystem    Size   Used  Avail Capacity  Mounted on
/dev/ad0s1h    87G   1.9G    78G     2%    /data
/dev/ad0s1f   1.2G   224M   872M    20%    /usr

mkdir /data/ports
mkdir /usr/ports

mount -t union /data/ports /usr/ports
cd /usr
cvs checkout ports
*optionally wait a day*
cvs update -dP ports

run systat -v or top and watch the machine do nothing. wait 15 minutes.
switch to another vty/login/whatever and try to start anything else.
Wait another 15 minutes, when the command hasn't even started decide the
machine is hung. Reboot, when the machine comes up umount the unionfsen
and try again directly in /data.
>Fix:
n/a
>Release-Note:
>Audit-Trail:
>Unformatted: