ZFS stalled after some mirror disks were lost

[Adam Vande More]

On Mon, Oct 2, 2017 at 1:12 PM, Ben RUBSON <ben.rubson at gmail.com> wrote:

> Hi,
>
> On a FreeBSD 11 server, the following online/healthy zpool :
>
> home
>   mirror-0
>     label/local1
>     label/local2
>     label/iscsi1
>     label/iscsi2
>   mirror-1
>     label/local3
>     label/local4
>     label/iscsi3
>     label/iscsi4
> cache
>   label/local5
>   label/local6
>
> A sustained read throughput of 180 MB/s, 45 MB/s on each iscsi disk
> according to "zpool iostat", nothing on local disks (strange but I
> noticed that IOs always prefer iscsi disks to local disks).
> No write IOs.
>
> Let's disconnect all iSCSI disks :
> iscsictl -Ra
>
> Expected behavior :
> IO activity flawlessly continue on local disks.
>

Perhaps I'm misunderstanding your setup, but my expected behavior would be
exactly what you see.  I think you'd need something more along the lines of:

home
  mirror
    label/local1
    label/iscsi1
  mirror
    label/local2
    label/iscsi2
etc...
-- 
Adam