FreeBSD 9-RELEASE zpool Out of File Descriptors ??
Fleuriot Damien
ml at my.gd
Wed May 15 16:03:44 UTC 2013
On May 15, 2013, at 5:56 PM, Outback Dingo <outbackdingo at gmail.com> wrote:
>
>
>
> On Wed, May 15, 2013 at 11:49 AM, Fleuriot Damien <ml at my.gd> wrote:
>
> On May 15, 2013, at 2:46 PM, Outback Dingo <outbackdingo at gmail.com> wrote:
>
> > On Wed, May 15, 2013 at 8:34 AM, Ronald Klop <ronald-freebsd8 at klop.yi.org>wrote:
> >
> >> On Wed, 15 May 2013 14:27:10 +0200, Outback Dingo <outbackdingo at gmail.com>
> >> wrote:
> >>
> >> On Wed, May 15, 2013 at 8:22 AM, Ronald Klop <ronald-freebsd8 at klop.yi.org
> >>>> **wrote:
> >>>
> >>> On Wed, 15 May 2013 14:13:32 +0200, Outback Dingo <
> >>>> outbackdingo at gmail.com>
> >>>> wrote:
> >>>>
> >>>> So it seems a new deployment we just built with zfsonroot mirror and a
> >>>>
> >>>>> 48TB
> >>>>> master pool is already out of File Descriptors???
> >>>>>
> >>>>> pool: master
> >>>>> state: ONLINE
> >>>>> status: The pool is formatted using a legacy on-disk format. The pool
> >>>>> can
> >>>>> still be used, but some features are unavailable.
> >>>>> action: Upgrade the pool using 'zpool upgrade'. Once this is done, the
> >>>>> pool will no longer be accessible on software that does not
> >>>>> support
> >>>>> feature
> >>>>> flags.
> >>>>> scan: none requested
> >>>>> config:
> >>>>>
> >>>>> NAME STATE READ WRITE CKSUM
> >>>>> master ONLINE 0 0 0
> >>>>> raidz3-0 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN01 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN03 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN04 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN05 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN07 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN08 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN09 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN10 ONLINE 0 0 0
> >>>>> raidz3-1 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN11 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN12 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN13 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN14 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN15 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN16 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN17 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN18 ONLINE 0 0 0
> >>>>> raidz3-2 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN19 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN20 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN21 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN22 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN23 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN24 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN26 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN27 ONLINE 0 0 0
> >>>>> raidz3-3 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN29 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN30 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN32 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN33 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN35 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN36 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN37 ONLINE 0 0 0
> >>>>> multipath/SATA_LUN38 ONLINE 0 0 0
> >>>>> logs
> >>>>> multipath/SATA_LUN06 ONLINE 0 0 0
> >>>>> cache
> >>>>> multipath/SATA_LUN02 ONLINE 0 0 0
> >>>>>
> >>>>> errors: No known data errors
> >>>>> pool: tank
> >>>>> state: ONLINE
> >>>>> status: The pool is formatted using a legacy on-disk format. The pool
> >>>>> can
> >>>>> still be used, but some features are unavailable.
> >>>>> action: Upgrade the pool using 'zpool upgrade'. Once this is done, the
> >>>>> pool will no longer be accessible on software that does not
> >>>>> support
> >>>>> feature
> >>>>> flags.
> >>>>> scan: none requested
> >>>>> config:
> >>>>>
> >>>>> NAME STATE READ WRITE CKSUM
> >>>>> tank ONLINE 0 0 0
> >>>>> mirror-0 ONLINE 0 0 0
> >>>>> da34p3 ONLINE 0 0 0
> >>>>> da35p3 ONLINE 0 0 0
> >>>>>
> >>>>> errors: No known data errors
> >>>>>
> >>>>> while compiling a kernel, buildworld worked and installed ok
> >>>>>
> >>>>> lex -t
> >>>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/****
> >>>>> aicasm/aicasm_scan.l
> >>>>>
> >>>>> aicasm_scan.c
> >>>>>>
> >>>>>> lex -t -Pmm
> >>>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/****
> >>>>>
> >>>>> aicasm/aicasm_macro_scan.l
> >>>>>
> >>>>> aicasm_macro_scan.c
> >>>>>>
> >>>>>> rm -f .depend_aicasm
> >>>>> mkdep -f .depend_aicasm -a -I.
> >>>>> -I/usr/src/sys/modules/****aic7xxx/aicasm/../../../dev/****
> >>>>> aic7xxx/aicasm
> >>>>> -std=gnu99
> >>>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/****
> >>>>> aicasm/aicasm.c
> >>>>> /usr/src/sys/modules/aic7xxx/****aicasm/../../../dev/aic7xxx/****
> >>>>>
> >>>>> aicasm/aicasm_symbol.c
> >>>>> aicasm_gram.c aicasm_macro_gram.c aicasm_scan.c aicasm_macro_scan.c
> >>>>> Out of file descriptors
> >>>>> *** [.depend_aicasm] Error code 2
> >>>>>
> >>>>> Stop in /usr/src/sys/modules/aic7xxx/****aicasm.
> >>>>>
> >>>>> *** [buildkernel] Error code 1
> >>>>>
> >>>>> Stop in /usr/src.
> >>>>> *** [buildkernel] Error code 1
> >>>>>
> >>>>>
> >>>>
> >>>> Do you know what a file descriptor is?
> >>>>
> >>>> There is something about it here: http://www.freebsd.org/doc/en/****<http://www.freebsd.org/doc/en/**>
> >>>> books/handbook/configtuning-****kernel-limits.html<http://www.**
> >>>> freebsd.org/doc/en/books/**handbook/configtuning-kernel-**limits.html<http://www.freebsd.org/doc/en/books/handbook/configtuning-kernel-limits.html>>(search
> >>>> for 'file descriptor')
> >>>> Or more info in here: https://www.google.nl/search?****<https://www.google.nl/search?**>
> >>>> q=freebsd+Out+of+file+****descriptors<https://www.**
> >>>> google.nl/search?q=freebsd+**Out+of+file+descriptors<https://www.google.nl/search?q=freebsd+Out+of+file+descriptors>
> >>>>>
> >>>>
> >>>>
> >>>> It mainly says that you have more files open than your system is
> >>>> configured to allow. This is often produced by a bug in a program which
> >>>> does not close some files properly.
> >>>>
> >>>>
> >>>> Yes I do.... but this is a brand new zfsonroot with barely any data on
> >>> it
> >>> and
> >>>
> >>> sysctl -a | grep kern.openfiles
> >>> kern.openfiles: 68
> >>> root@:/master/builder # sysctl -a | grep kern.maxfiles
> >>> kern.maxfiles: 24600
> >>> kern.maxfilesperproc: 11095
> >>>
> >>> the openfiles and maxfiles seems to be plenty, but im out of descriptors,
> >>> i
> >>> used to see this back in the 4.x days when you could format
> >>> ufs with larger inodes, but zfs ??? really?
> >>>
> >>
> >>
> >> Is this kern.openfiles when idle or during your buildkernel?
> >> Did you check 'ulimit -a'?
> >> You say installworld worked ok while you are just now doing buildkernel.
> >> Are your kernel and world out of sync?
> >>
> >>
> > well, i was upgrading FreeBSD 9-RELEASE to CURRENT, did a make buildworld,
> > then a make installworld, when i went to do a make buildkernel
> > thats the error i got, so id say, yes it is "out of sync"
>
>
> Wow wow wow, hold on a sec here.
>
> You're installing your new world before the kernel ?
>
>
> I've had no trouble upgrading several boxes from 8-STABLE to 10.0-CURRENT, following the regular procedure as described at:
> http://www.freebsd.org/doc/handbook/makeworld.html
>
>
> Note that the correct order is:
> - buildworld (or kernel-toolchain if you only want to test the kernel and not do the whole world)
> - buildkernel
> - installkernel
> - reboot
> - mergemaster -p (preferably in single user, I've always run it multiuser w/o problems)
> - installworld
> - mergemaster
> - reboot
>
> You don't want to run the new world on the old kernel.
>
> yeah i know better, seems when i went to install the old kernel the build had failed, i can get through it from here
>
> it was odd because the CURRENT kernel failed to build with a yyparse error, until after i had completed a buildworld
>
> the other node i did is fine..........
>
Perhaps their sources weren't at the same revision level ?
Anyway, you wanna reinstall a 9.x world from the ISO now, then update sources and redo things in order ;)
As a sidenote, if your root is on UFS, you can use nextboot to failsafe-try your new kernel.
More information about the freebsd-fs
mailing list