gmirror balance model "prefer" doesnt work properly

[Mark Johnston]

On Sat, Aug 15, 2020 at 11:34:07AM +0200, Wojciech Puchar wrote:
> ada0p3 is partition on SSD, ada3p3 is partition on magnetic disk.
> 
> 
> 
> Geom name: root
> State: COMPLETE
> Components: 2
> Balance: prefer
> Slice: 2097152
> Flags: NONE
> GenID: 0
> SyncID: 1
> ID: 3736675121
> Type: AUTOMATIC
> Providers:
> 1. Name: mirror/root
>     Mediasize: 107374181888 (100G)
>     Sectorsize: 512
>     Stripesize: 4096
>     Stripeoffset: 0
>     Mode: r1w1e1
> Consumers:
> 1. Name: ada0p3
>     Mediasize: 107374182400 (100G)
>     Sectorsize: 512
>     Stripesize: 0
>     Stripeoffset: 525139968
>     Mode: r1w1e1
>     State: ACTIVE
>     Priority: 1
>     Flags: (null)
>     GenID: 0
>     SyncID: 1
>     ID: 3655477467
> 2. Name: ada3p3
>     Mediasize: 107374182400 (100G)
>     Sectorsize: 512
>     Stripesize: 4096
>     Stripeoffset: 0
>     Mode: r1w1e1
>     State: ACTIVE
>     Priority: 0
>     Flags: NONE
>     GenID: 0
>     SyncID: 1
>     ID: 3836554217
> 
> 
> according to manual reads should be preferred from device of higher 
> priority. i've set priority 1 for ssd, 0 for hdd. but reads are from hdd 
> only.
> 
> tried reverse - 0 for ssd, 1 for hdd - too reads are from hdd.
> 
> looking in code seems like it is not implemented at all
> 
>          LIST_FOREACH(disk, &sc->sc_disks, d_next) {
>                  if (disk->d_state == G_MIRROR_DISK_STATE_ACTIVE)
>                          break;
>          }
> 
> 
> just take first active disk.
> 
> why?

The highest priority disk is supposed to be first in the list.  However,
the list is only sorted once the mirror is started, which is arguably
the bug.  If you restart the mirror after adjusting the disk priorities,
I'd expect the "prefer" algorithm to work properly.