Post 9.1 stable file system problems

[Konstantin Belousov]

On Tue, Jan 01, 2013 at 02:05:11AM +0100, Dominic Fandrey wrote:
> On 01/01/2013 01:49, Dominic Fandrey wrote:
> > On 01/01/2013 01:29, Chris Rees wrote:
> >> On 1 Jan 2013 00:01, "Dominic Fandrey" <kamikaze at bsdforen.de> wrote:
> >>>
> >>> I have a Tinderbox that I just updated to the current RELENG_9.
> >>> Following the update build times for packages have increased by a
> >>> factor between 5 and 20. I.e. I have packages that used to build in
> >>> 5 minutes and now take an hour.
> >>>
> >>> I'm suspecting the file system ever since I saw that the majority of CPU
> >>> load was caused by ls when I looked at top (more than 2 minutes of CPU
> >>> time were counted that moment). The majority of the time most of the CPU
> >>> load is caused by bsdtar, pkg_add, qmake-qt4, etc. Without exception
> >>> tools that access a lot of files.
> >>>
> >>> The file system on which packages are built is nullfs mounted from
> >>> an async mounted UFS. I turned async off, to no avail.
> >>>
> >>> /usr/src/UPDATING says that there were nullfs optimisations. So I
> >>> think this is where the problem originates. I might hack the tinderbox to
> >>> use 'ln -s' or set it up for NFS to verify this.
> >>
> >> Is your kernel newer than the Jail?  The converse causes problems.
> > 
> > I ran makeJail for all jails after updating.
> > 
> > I also seem to have similar problems when building in the host-system.
> > The unzip for openjdk-7 has just passed the 11 minutes CPU time mark.
> > On my notebook it takes less than 10 seconds.
> 
> Just set WRKOBJDIRPREFIX to a tmpfs on the Tinderbox host system
> and the extract takes less than a second. Originally WRKOBJDIRPREFIX
> also pointed to a nullfs mount.
> 
> Afterwards I pointed WRKOBJDIRPREFIX to a UFS file system (without
> nullfs involvement). The entire make extract took 20s.
> 
> So still faster by at least factor 30 than running it on a nullfs mount
> (I eventually SIGINTed so I don't know how long it would've run).

Start providing some useful debugging information ?

At least dmesg, mount -v and sysctl kern.maxvnodes,
'sysctl vfs | grep vnodes' outputs.

What is shown when you press ^T while slow process runs on nullfs ?
Was the ^C reaction by terminating the process instant ?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 834 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20130101/ccf6c288/attachment.sig>