Unlistable, Undeletable File in old-jailed clientmqueue folder

[Trond Endrestøl]

On Sat, 8 Mar 2014 23:48+1030, Gitsnik wrote:

> Hi all,
> 
> Briefly, there are random-named files in a couple of clientmqueue
> directories that I can not delete. These directories were for 32bit
> jails prior to re-installing my base system to amd64, so I know they're
> not locked by any "normal" process.
> 
> My apologies for the length, hopefully too many details is a good thing.
> 
> 9.1-RELEASE-p10 GENERIC amd64, although I've had this problem the entire
> 9.x branch it is entirely possible it is because I re-installed my root
> partition to go from 9.0-RELEASE x86 to amd64. During the install my
> zpool "raidpack" was left alone, except to import it after the
> installation, it has since been upgraded and is now running the current
> version 28 for my install. "raidpack" pool is a raidz1-0 with 3 x 2TB
> drives, is scrubbed weekly and has so far had no issues bar this.
> 
> Previously I was running jails directly from the zpool, post reinstall I
> have started using a fresh directory (prev: /raidpack/jails, cur:
> /raidpack/64bitjails). The difference is simple enough - "jails" was the
> 32bit jails directory.
> 
> My question/ problem:
> 
> # ls -l
> total 179

> drwxrwx---  2 root  wheel  3 Oct  3 21:26 clientmqueue
                             ^
3 links to this directory? --+

ZFS is a bit peculiar as it records the number of links when dealing 
with directories.

Assume for a moment that your clientmqueue directory really is empty. 
It should then have a link count of 2; the clientmqueue link and the 
dot link (.) inside the clientmqueue directory.

Back to reality, why are there 3 links, not two, to an otherwise empty 
directory?

Chances are that the qfq8G32oib058119 entry has somehow been 
transformed into a directory link pointing to dot.

This is just speculation on my part. Maybe some bad memory got written 
to disk at some point, as ZFS assumes memory (RAM) is 100 % healthy at 
all times.

You should make good backups of the pool's filesystems, export the 
pool and try booting from the latest stable/9 snapshot. Try to import 
the pool without mounting any of the filesystems (zpool import -N) and 
run zpool scrub. If all goes well, export the pool, reboot, and import 
the pool. Worst-case scenario is to re-create the pool and restoring 
everything from backups.

> # cd clientmqueue
> # ls
> qfq8G32oib058119
> # ls -la
> ls: qfq8G32oib058119: No such file or directory
> total 181
> drwxrwx---  2 root  wheel  3 Oct  3 21:26 .
> drwxr-xr-x  3 root  wheel  3 Mar  8 23:28 ..
> # rm qfq8G32oib058119
> rm: qfq8G32oib058119: No such file or directory
> #
> 
> No matter what I do, I can't seem to delete this file. fstat and getfacl
> both report "No such file or directory". The following information is
> shown for the folder itself:
> 
> # getfacl clientmqueue
> # file: clientmqueue
> # owner: root
> # group: wheel
>             owner@:--------------:------:deny
>             owner@:rwxp---A-W-Co-:------:allow
>             group@:--------------:------:deny
>             group@:rwxp----------:------:allow
>          everyone@:rwxp---A-W-Co-:------:deny
>          everyone@:------a-R-c--s:------:allow
> # ls -lTdo clientmqueue
> drwxrwx---  2 root  wheel  - 3 Oct  3 21:26:36 2013 clientmqueue
> #
> 
> Thanks for reading through, any help is much appreciated.

-- 
+-------------------------------+------------------------------------+
| Vennlig hilsen,               | Best regards,                      |
| Trond Endrestøl,              | Trond Endrestøl,                   |
| IT-ansvarlig,                 | System administrator,              |
| Fagskolen Innlandet,          | Gjøvik Technical College, Norway,  |
| tlf. mob.   952 62 567,       | Cellular...: +47 952 62 567,       |
| sentralbord 61 14 54 00.      | Switchboard: +47 61 14 54 00.      |
+-------------------------------+------------------------------------+