ZFS: Panic when attempting to delete certain data

Josh Beard josh at signalboxes.net
Tue Nov 27 21:47:31 UTC 2012


On Tue, Nov 27, 2012 at 2:13 PM, Andriy Gapon <avg at freebsd.org> wrote:

> on 27/11/2012 22:32 Josh Beard said the following:
> >
> >
> > On Tue, Nov 27, 2012 at 11:48 AM, Andriy Gapon <avg at freebsd.org
> > <mailto:avg at freebsd.org>> wrote:
> >
> >     on 27/11/2012 20:25 Josh Beard said the following:
> >     > Hello,
> >     >
> >     > I have a system that I can consistently reproduce a panic on when
> trying to
> >     > delete certain data.  The data is data that was rsynced from
> another system
> >     > - nothing terribly unique.  This has been ongoing from several
> months,
> >     > starting with 9.0-RELEASE and now running 9.1-RC3.
> >     >
> >     > I can't find anything in common with the files that I can trigger
> the
> >     > panics with.  One is a simple gzipped archive where some are plain
> text.
> >     >  Strangely, I can only reproduce it with data that was rsynced
> from that
> >     > particular system (which is a Mac).
> >
> >     Josh,
> >
> >     I am collecting these cases, thank you for another one.
> >     I had an interesting investigation of
> >     http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/173747
> >     Unfortunately, for some reason the whole conversation stayed private.
> >     I see that also opened a PR earlier:
> >     http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/170238
> >
> >     Could you please provide the following info?
> >     >From kgdb:
> >     - list in frame 7 (zfs_freebsd_remove), so that I can see the code
> line
> >     - local variables from frame 7 (info local)
> >
> >
> >
> > Andriy,
> >
> > Thanks for your quick response.  I've never used kgdb, so forgive my
> ignorance
> > here.  Is this what you're looking for?  If not, if you could you
> elaborate on
> > those?
> >
> > #7  0xffffffff80ebd45a in zfs_freebsd_remove (ap=Variable "ap" is not
> available.
> > ) at
> >
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1855
> > 1855                    dmu_tx_hold_sa(tx, xzp->z_sa_hdl, B_FALSE);
> >
> >
> > (kgdb) list zfs_freebsd_remove
> > 5796            struct vop_remove_args /* {
> > 5797                    struct vnode *a_dvp;
> > 5798                    struct vnode *a_vp;
> > 5799                    struct componentname *a_cnp;
> > 5800            } */ *ap;
> > 5801    {
> > 5802
> > 5803            ASSERT(ap->a_cnp->cn_flags & SAVENAME);
> > 5804
> > 5805            return (zfs_remove(ap->a_dvp, ap->a_cnp->cn_nameptr,
>
> Not quite :-)
> frame 7
> list
>
> > (kgdb) info frame 7
> > Stack frame at 0xffffff8466a6a920:
> >  rip = 0xffffffff80ebd45a in zfs_freebsd_remove
> >
> (/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1855);
> > saved rip 0xffffffff8081cf13
> >  called by frame at 0xffffff8466a6a940, caller of frame at
> 0xffffff8466a6a7a0
> >  source language c.
> >  Arglist at 0xffffff8466a6a910, args: ap=Variable "ap" is not available.
>
> frame 7
> info local
>

Thanks!  Here we go:

(kgdb) frame 7
#7  0xffffffff80ebd45a in zfs_freebsd_remove (ap=Variable "ap" is not
available.
) at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1855
1855                    dmu_tx_hold_sa(tx, xzp->z_sa_hdl, B_FALSE);
(kgdb) list
1850                &xattr_obj, sizeof (xattr_obj));
1851            if (error == 0 && xattr_obj) {
1852                    error = zfs_zget(zfsvfs, xattr_obj, &xzp);
1853                    ASSERT3U(error, ==, 0);
1854                    dmu_tx_hold_sa(tx, zp->z_sa_hdl, B_TRUE);
1855                    dmu_tx_hold_sa(tx, xzp->z_sa_hdl, B_FALSE);
1856            }
1857
1858            mutex_enter(&zp->z_lock);
1859            if ((acl_obj = zfs_external_acl(zp)) != 0 && may_delete_now)
(kgdb) frame 7
#7  0xffffffff80ebd45a in zfs_freebsd_remove (ap=Variable "ap" is not
available.
) at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1855
1855                    dmu_tx_hold_sa(tx, xzp->z_sa_hdl, B_FALSE);
(kgdb) info local
No locals.
(kgdb)



> >     Also, for one of the files that trigger the problem:
> >     - ls -i to obtain its inode number
> >     - zdb -ddddd <dataset name> <inode number>
> >
> >
> > # ls -i kyofilter\ v2.2.pax.gz  (this is a symlink.  the file that it's
> linked
> > to does *not* panic the system if I try to delete it).
>
> Hmm, then could you please rather do 'ls -Pi kyofilter\ v2.2.pax.gz' and
> then
> use that value for the zdb command?
>

# ls -Pi /DSDK12_NHR.pax.gz  (symlink to ../Archive.pax.gz)
249868
./Imaging/Packages/DSDK12_NHR_2012-02-23.pkg/Contents/Resources/DSDK12_NHR.pax.gz

# zdb -ddddd store/tdxs1 249868
Dataset store/tdxs1 [ZPL], ID 109, cr_txg 35014, 1.33T, 1106389 objects,
rootbp DVA[0]=<0:8000204400:400> DVA[1]=<0:30800644400:400> [L0 DMU objset]
fletcher4 lzjb LE contiguous unique double size=800L/200P
birth=1167710L/1167710P fill=1106389
cksum=1966704b59:757ae6cb615:134bfd597bca9:254b2ee348393d

    Object  lvl   iblk   dblk  dsize  lsize   %full  type
    249868    1    16K    512      0    512    0.00  ZFS plain file
                                        201   bonus  System attributes
        dnode flags: USERUSED_ACCOUNTED
        dnode maxblkid: 0
        path
 /tech/2012-09-14-01-00/Imaging/Packages/DSDK12_NHR_2012-02-23.pkg/Contents/Resources/DSDK12_NHR.pax.gz
        uid     300002
        gid     80
        atime   Tue Nov 27 14:43:00 2012
        mtime   Thu Feb 23 08:59:21 2012
        ctime   Fri Sep 14 01:12:37 2012
        crtime  Fri Sep 14 01:11:50 2012
        gen     81430
        mode    120755
        size    17
        parent  249866
        links   1
        pflags  40800000104
        xattr   230
Indirect blocks:




>
> > 247126 kyofilter v2.2.pax.gz
> >
> > # zdb -ddddd store/tdxs1 247126
> > Dataset store/tdxs1 [ZPL], ID 109, cr_txg 35014, 1.33T, 1106389 objects,
> rootbp
> > DVA[0]=<0:80001a2400:400> DVA[1]=<0:30800610000:400> [L0 DMU objset]
> fletcher4
> > lzjb LE contiguous unique double size=800L/200P birth=1166838L/1166838P
> > fill=1106389 cksum=19391f0f67:78eb24a9cca:1439005549d01:275015332d1bdf
> >
> >     Object  lvl   iblk   dblk  dsize  lsize   %full  type
> >     247126    1    16K    512      0    512    0.00  ZFS plain file
> >                                         201   bonus  System attributes
> >         dnode flags: USERUSED_ACCOUNTED
> >         dnode maxblkid: 0
> >         path
>  /tech/2012-09-14-01-00/Drivers/Kyocera/.old/C2126.old/Kyocera OS
> > X 10.5+ Web build 2011.01.27.mpkg/Contents/Packages/Kyocera OS X
> > subinstaller.mpkg/Contents/Packages/kyofilter
> > v2.2.pkg/Contents/Resources/kyofilter v2.2.pax.gz
> >         uid     1001
> >         gid     80
> >         atime   Tue Nov 27 13:27:57 2012
> >         mtime   Tue Jul 12 14:17:16 2011
> >         ctime   Fri Sep 14 01:05:23 2012
> >         crtime  Fri Sep 14 01:04:11 2012
> >         gen     81338
> >         mode    120755
> >         size    17
> >         parent  247122
> >         links   1
> >         pflags  40800000104
> >         xattr   155
>
> --
> Andriy Gapon
>


More information about the freebsd-fs mailing list