Re: git: 75412a521f60 - main - Hyper-V: vPCI: Prepopulate device bars

On Sat, Nov 27, 2021 at 06:55:54AM +0000, Wei Hu wrote:
> The branch main has been updated by whu:
> 
> URL: https://cgit.FreeBSD.org/src/commit/?id=75412a521f60d4b0393c730ffb284e7c6ff9d2de
> 
> commit 75412a521f60d4b0393c730ffb284e7c6ff9d2de
> Author:     Wei Hu <whu@FreeBSD.org>
> AuthorDate: 2021-11-27 06:42:34 +0000
> Commit:     Wei Hu <whu@FreeBSD.org>
> CommitDate: 2021-11-27 06:42:34 +0000
> 
>     Hyper-V: vPCI: Prepopulate device bars
>     
>     In recent Hyper-V releases on Windows Server 2022, vPCI code does not
>     initialize the last 4 bit of device bar registers. This behavior change
>     could result weird problems cuasing PCI code failure when configuring
>     bars.
>     
>     Just write all 1's to those bars whose probed values are not the same
>     as current read ones. This seems to make Hyper-V vPCI and
>     pci_write_bar() to cooperate correctly on these releases.
Could you please elaborate both on the problem and on the solution more?

It really makes not a lot of sense to me.  Four low bits of BARs are
space indicator, and for memory bar, 32/64 bitness and prefetch bit.
It is some magic that, following your description, these values are
wrong but writing 1's to them validate the value.

Were other OSes affected by the bug? Do they use the same workaround?

>     
>     Reported by:    khng@freebsd.org
>     Tested by:      khng@freebsd.org
>     MFC after:      2 weeks
>     Sponsored by:   Microsoft
> ---
>  sys/dev/hyperv/pcib/vmbus_pcib.c | 43 ++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 43 insertions(+)
> 
> diff --git a/sys/dev/hyperv/pcib/vmbus_pcib.c b/sys/dev/hyperv/pcib/vmbus_pcib.c
> index 72e430c946db..c7df32044678 100644
> --- a/sys/dev/hyperv/pcib/vmbus_pcib.c
> +++ b/sys/dev/hyperv/pcib/vmbus_pcib.c
> @@ -1356,6 +1356,47 @@ _hv_pcifront_write_config(struct hv_pci_dev *hpdev, int where, int size,
>  	}
>  }
>  
> +/*
> + * The vPCI in some Hyper-V releases do not initialize the last 4
> + * bit of BAR registers. This could result weird problems causing PCI
> + * code fail to configure BAR correctly.
> + *
> + * Just write all 1's to those BARs whose probed values are not zero.
> + * This seems to make the Hyper-V vPCI and pci_write_bar() to cooperate
> + * correctly.
> + */
> +
> +static void
> +vmbus_pcib_prepopulate_bars(struct hv_pcibus *hbus)
> +{
> +	struct hv_pci_dev *hpdev;
> +	int i;
> +
> +	mtx_lock(&hbus->device_list_lock);
> +	TAILQ_FOREACH(hpdev, &hbus->children, link) {
> +		for (i = 0; i < 6; i++) {
> +			/* Ignore empty bar */
> +			if (hpdev->probed_bar[i] == 0)
> +				continue;
> +
> +			uint32_t bar_val = 0;
> +
> +			_hv_pcifront_read_config(hpdev, PCIR_BAR(i),
> +			    4, &bar_val);
> +
> +			if (hpdev->probed_bar[i] != bar_val) {
> +				if (bootverbose)
> +					printf("vmbus_pcib: initialize bar %d "
> +					    "by writing all 1s\n", i);
> +
> +				_hv_pcifront_write_config(hpdev, PCIR_BAR(i),
> +				    4, 0xffffffff);
> +			}
> +		}
> +	}
> +	mtx_unlock(&hbus->device_list_lock);
> +}
> +
>  static void
>  vmbus_pcib_set_detaching(void *arg, int pending __unused)
>  {
> @@ -1479,6 +1520,8 @@ vmbus_pcib_attach(device_t dev)
>  	if (ret)
>  		goto vmbus_close;
>  
> +	vmbus_pcib_prepopulate_bars(hbus);
> +
>  	hbus->pci_bus = device_add_child(dev, "pci", -1);
>  	if (!hbus->pci_bus) {
>  		device_printf(dev, "failed to create pci bus\n");