git: c6f3076d4405 - main - Move the GICv2m msi handling to the parent

[Jessica Clarke]

On 28 Sep 2021, at 01:32, Marcin Wojtas <mw at semihalf.com> wrote:
> 
> Hi Andrew,
> 
> 
> wt., 14 wrz 2021 o 12:42 Andrew Turner <andrew at freebsd.org> napisał(a):
>> 
>> The branch main has been updated by andrew:
>> 
>> URL: https://cgit.FreeBSD.org/src/commit/?id=c6f3076d44055f7b02467ce074210f73d0ce0ef6
>> 
>> commit c6f3076d44055f7b02467ce074210f73d0ce0ef6
>> Author:     Andrew Turner <andrew at FreeBSD.org>
>> AuthorDate: 2021-09-01 09:39:01 +0000
>> Commit:     Andrew Turner <andrew at FreeBSD.org>
>> CommitDate: 2021-09-14 07:24:52 +0000
>> 
>>    Move the GICv2m msi handling to the parent
>> 
>>    This is in preperation for adding support for the GICv2m driver as a
>>    child of the GICv3 driver.
>> 
>>    PR:             258136
>>    Reported by:    trasz
>>    Sponsored by:   The FreeBSD Foundation
>>    Differential Revision: https://reviews.freebsd.org/D31767
>> ---
>> sys/arm/arm/gic.c        | 296 ++++++++++++++++++++++++++++++-----------------
>> sys/arm/arm/gic.h        |   8 +-
>> sys/arm/arm/gic_common.h |   4 +
>> 3 files changed, 195 insertions(+), 113 deletions(-)
>> 
>> diff --git a/sys/arm/arm/gic.c b/sys/arm/arm/gic.c
>> index 1851e69644ed..bd34e92b9e28 100644
>> --- a/sys/arm/arm/gic.c
>> +++ b/sys/arm/arm/gic.c
>> @@ -501,6 +501,56 @@ arm_gic_read_ivar(device_t dev, device_t child, int which, uintptr_t *result)
>>                    ("arm_gic_read_ivar: Invalid bus type %u", sc->gic_bus));
>>                *result = sc->gic_bus;
>>                return (0);
>> +       case GIC_IVAR_MBI_START:
>> +               *result = sc->sc_spi_start;
>> +               return (0);
>> +       case GIC_IVAR_MBI_COUNT:
>> +               *result = sc->sc_spi_count;
>> +               return (0);
>> +       }
>> +
>> +       return (ENOENT);
>> +}
>> +
>> +static int
>> +arm_gic_write_ivar(device_t dev, device_t child, int which, uintptr_t value)
>> +{
>> +       struct arm_gic_softc *sc;
>> +
>> +       sc = device_get_softc(dev);
>> +
>> +       switch(which) {
>> +       case GIC_IVAR_HW_REV:
>> +       case GIC_IVAR_BUS:
>> +               return (EINVAL);
>> +       case GIC_IVAR_MBI_START:
>> +               /*
>> +                * GIC_IVAR_MBI_START must be set once and first. This allows
>> +                * us to reserve the registers when GIC_IVAR_MBI_COUNT is set.
>> +                */
>> +               MPASS(sc->sc_spi_start == 0);
>> +               MPASS(sc->sc_spi_count == 0);
> 
> This patch breaks GICv2m on all Marvell Armada 7k8k and CN913x. After
> reverting it works as expected. I tried removing the above 2 asserts +
> the one in line 540 - instead of init it fails later, during PCIE
> endpoint msix configuration:
> 
> em0: Using 1024 TX descriptors and 1024 RX descriptors
> em0: Using 2 RX queues 2 TX queues
> panic: mtx_lock() of spin mutex GIC lock @
> /home/mw/current/sys/arm/arm/gic.c:1145
> cpuid = 0
> time = 1
> KDB: stack backtrace:
> db_trace_self() at db_trace_self
> db_trace_self_wrapper() at db_trace_self_wrapper+0x30
> vpanic() at vpanic+0x184
> panic() at panic+0x44
> __mtx_lock_flags() at __mtx_lock_flags+0x1a8
> arm_gic_alloc_msix() at arm_gic_alloc_msix+0x40
> intr_alloc_msix() at intr_alloc_msix+0x190
> generic_pcie_acpi_alloc_msix() at generic_pcie_acpi_alloc_msix+0x78
> pci_alloc_msix_method() at pci_alloc_msix_method+0x1a8
> iflib_device_register() at iflib_device_register+0xae4
> iflib_device_attach() at iflib_device_attach+0xd0
> device_attach() at device_attach+0x410
> device_probe_and_attach() at device_probe_and_attach+0x7c
> bus_generic_attach() at bus_generic_attach+0x18
> pci_attach() at pci_attach+0xe8
> device_attach() at device_attach+0x410
> device_probe_and_attach() at device_probe_and_attach+0x7c
> bus_generic_attach() at bus_generic_attach+0x18
> device_attach() at device_attach+0x410
> device_probe_and_attach() at device_probe_and_attach+0x7c
> bus_generic_new_pass() at bus_generic_new_pass+0xec
> bus_generic_new_pass() at bus_generic_new_pass+0xd0
> bus_generic_new_pass() at bus_generic_new_pass+0xd0
> bus_set_pass() at bus_set_pass+0x8c
> mi_startup() at mi_startup+0x12c
> virtdone() at virtdone+0x6c
> KDB: enter: panic
> [ thread pid 0 tid 100000 ]
> Stopped at      kdb_enter+0x44: undefined       f900c11f
> 
> Any ideas?

Change all the mtx_(un)lock(&sc->mutex) to be the _spin versions.

Jess