GEOM hangover disables NFS
Doug Rabson
dfr at rabson.org
Fri Nov 14 02:36:54 PST 2008
On 14 Nov 2008, at 10:05, Ivan Voras wrote:
> Steve Kargl wrote:
>> On three nodes in my cluster (nodes n17, n18, and n19), I had
>> GEOM use /dev/ad4s1e for tests with gmirror and ggated/ggatec.
>> I found that GEOM was insufficient for my needs and decided
>> to return the 3 partitions to NFS-exported partitions. It seems
>> that once GEOM touches a partition, the partition can no longer
>> be used by NFS.
>>
>> I'll illustrute the problem with n17:/dev/ad4s1e. In what follows,
>> n10 is the master node. Both n10 and n17 have brand new worlds
>> and kernels from about 45 minutes ago.
>>
>> n10:kargl[203] ssh n17
>> n17:kargl[201] df
>> Filesystem 1M-blocks Used Avail Capacity Mounted on
>> /dev/ad4s1a 247 104 123 46% /
>> devfs 0 0 0 100% /dev
>> /dev/ad4s1e 222780 0 204958 0% /data
>> /dev/ad4s1d 3962 182 3463 5% /usr
>> n10:/home 193947 92855 85576 52% /home
>> n10:/usr/local 19832 10494 7750 58% /usr/local
>>
>> n17:kargl[202] tail -1 /etc/exports
>> /data -alldirs node10 node21
>>
>> The above is after a 'newfs -U /dev/ad4s1e' and a reboot.
>>
>> n10:root[244] ls / | grep -E ^n
>> n11/ n12/ n13/ n14/ n15/ n16/ n17/ n18/ n19/ n20/ n21/
>>
>> n10:root[245] mount_nfs -o tcp n17:/data /n17
>> n10:root[246] mount -v | grep n17
>> n17:/data on /n17 (nfs, fsid 0eff000303000000)
>> n10:root[247] ls /n17
>> ls: /n17: Input/output error
>> n10:root[248] ls / | grep -E ^n
>> ls: n17: Input/output error
>> n11/ n12/ n13/ n14/ n15/ n16/ n18/ n19/ n20/ n21/
>>
>> n10:root[251] umount /n17
>> n10:root[252] ls / | grep -E ^n
>> n11/ n12/ n13/ n14/ n15/ n16/ n17/ n18/ n19/ n20/ n21/
>>
>> So, how does one exorcise GEOM from /dev/ad4s1e?
>
> Your message is very confusing. Here's some information that could
> help you:
>
> * "GEOM" is just a name for "system of handling disk-like devices in
> FreeBSD". If you "remove GEOM" (which would probably be hard to do and
> would involve heavily modifying the kernel source) you will have no
> access to your disk drives and other similar devices.
>
> * Using ggated/ggatec you can either:
> - mount the file system exactly once as rw and none as ro (doesn't
> matter if the file system is on local node or exported, the total is
> important)
> - mount the file system as ro any number of times and none as rw.
> (In other words, only one machine can have the file system mounted for
> reading and writing. Once a file system has been mounted as rw, no
> other
> machines can do anything with it. This has nothing to do with NFS as
> NFS
> operates on top of a mounted file system and inherits its rules.)
>
> * You can remove gmirror metadata on devices with "gmirror clean".
> ggated and ggatec don't store metadata on devices.
This was a bug in the NFS code which is now fixed. The bug had nothing
to do with GEOM.
More information about the freebsd-current
mailing list