kern/175323: Fail to use ZVOL as a gmirror component

[Alexei Volkov]

>Number:         175323
>Category:       kern
>Synopsis:       Fail to use ZVOL as a gmirror component
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Jan 15 12:00:00 UTC 2013
>Closed-Date:
>Last-Modified:
>Originator:     Alexei Volkov
>Release:        9.1-RELEASE
>Organization:
SoftLynx
>Environment:
FreeBSD fresh-inst 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243825: Tue Dec  4 09:23:
10 UTC 2012     root at farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64

>Description:
kernel: GEOM_MIRROR (error=45) while trying to use ZFS volumes (ZVOL) geoms as a geom_mirror component.
>How-To-Repeat:
# zfs create -V10G tank0/vol0
# diskinfo -v /dev/zvol/tank0/vol0
/dev/zvol/tank0/vol0
        512             # sectorsize
        10737418240     # mediasize in bytes (10G)
        20971520        # mediasize in sectors
        0               # stripesize
        0               # stripeoffset

# gmirror label mirror0 /dev/zvol/tank0/vol0
GEOM_MIRROR: Cannot add disk zvol/tank0/vol0 to mirror0 (error=45).
GEOM_MIRROR: Device mirror0 destroyed.

gmirror list is empty and gmirror list -a shows:

Geom name: mirror0
Consumers:
1. Name: zvol/tank0/vol0
   Mediasize: 10737418240 (10G)
   Sectorsize: 512
   Mode: r1w1e1

Right after the sequence above, the block device zvol/tank0/vol0 is not any more available for writing it (seems to be locked by geom_mirror)

# dd if=/dev/zero of=/dev/zvol/tank0/vol0
dd: /dev/zvol/tank0/vol0: Operation not permitted


And i was not able to release it.

# gmirror forget mirror0
gmirror: No such device: mirror0.

# gmirror clear /dev/zvol/tank0/vol0
Can't clear metadata on /dev/zvol/tank0/vol0: Operation not permitted.
gmirror: Not fully done.

# zfs destroy -f tank0/vol0
cannot destroy 'tank0/vol0': dataset is busy

The only way to destroy ZVOL is temporarily block geaom_morror loading during the boot process, reboot and destroy zvols.
>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: