4.x device failure?

[Sam]

> Are you calling disk_destroy()?  disk_invalidate() only frees
> the disk slice objects, it doesn't actually remove the disk
> object and cdevsw.  You also need to manually track opens in
> your disk driver and only call disk_destroy() once all references
> are closed.  What panic are you getting?
>
> Scott

OK - the panic was due to calling disk_invalidate, which calls
dsgone on the disk slice.  Later on the filesystem tries
to use this null pointer and falls over.  Bad me, I guess.

Now I don't call disk_invalidate until the device is closed
(when disk_destroy is also called).  The panic is gone, but
now I have a condition I can't recover from.

To recap, the AoE device is mounted and I unload the module.
The unload procedure fails outstanding bufs for the device
and waits for the device to be closed.  The close never comes.
Umounting the filesystem fails because he tries to open the
device, I see it's now gone, and return ENODEV.  "Device not configured"

What I'm looking for is a way to tell those who have the
device open, "this device is gone.  Clean up yourselves."

kldstat also hangs, presumably because I'm in the middle of
the unload procedure.

Sam