From nobody Fri May 12 17:09:40 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QHwHN3n5lz4BFsF for ; Fri, 12 May 2023 17:09:52 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QHwHN3D19z4T8h for ; Fri, 12 May 2023 17:09:52 +0000 (UTC) (envelope-from kevans@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1683911392; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i95UswbT5PymNpRyyK+QZmtCrTtHjYySV85dQ7Y7i0w=; b=oDCWUOgEmjC/H5YvJOEBq90t9sri1ANoH2DCOBA00ESW/IWzbQKRKYywkzcHqBO5RtbSTR evSVfslAKME0UG/DolBjK3Dj/KBchw59JYTq12RDT77h5xwS0U9/L2HejwkL2SMmnxUvVv G3egMM+tWRA9ARtindpfDAAOu0Ak9PSdz3lBEl0kgThjNPbzO7w5WcnzVAkKZOjZXe5kMy heO5/EoZfsQiKkUng3ucY4mFRkxHpbKMV3R+HhJ+rmaI2lonbBUAY966kz/qjhMNgLXmFV jeq2Iut7IeWyljyg1jW0bRtWTq/U6Gogj/2fqDG17yNxlsO0abRfqvGs4honDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1683911392; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i95UswbT5PymNpRyyK+QZmtCrTtHjYySV85dQ7Y7i0w=; b=J85VUsQOXIfvwTOFlupl79TcIS+b7kCxwEf1pUWnJZvfVgWRIZAt7f/Q7r/GfWFWurotbg iooy/rMrgEjBszfu5GsfkUdhUWTniI2S1ZHuj6et77AfAEA7Bar4qMcwbRzhAWi+U80HrF WzA2bPzAIf2VaK8oiueD9+PMWB1Xf9ZOyyjclf08823kcOq1fixfWqOV9Oy32I6kagoFcK Jkx6Dd7LObYUVHtfh5oiJCBKMVXpdT96go/uPGWSi3ys0FkvbUR/t3mhF5cezi7T0qudUP ZxfN4BzgnAVtg8GS3WDi/CX97iSnaV5XdMmMsOsSezEzfudx4D5VXcb1loxDhg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1683911392; a=rsa-sha256; cv=none; b=ISPoWOwM2BxoVoEjQB76eBJfQtXytamej+SExrK3IAHldg9AeC06oFqUFxuGTs1cIIKOVS EBDA0sTYYOaqNI8SdAheStZvyatdTTr01W/x4crB+B8zmrJYMUc1L6dGRPSsa/XbG1mikt WXiyQns+PdUJliQUb4+AgoVepkcWdsc1sIOA+QTDqvMPnZ3NxbchSjxteWozlG7kMvDB+L ws27dgvhqUdKGRylHT9zRWDcaS1fD+SXe6su3NriNt02c/pPlT3tmrHXrJ017r0xhCCd8n 5WniAFhXuH3xSXGKpPFFngcdAzqQw5uJdH3Z4Ezd50TdzMjeEWxWUW4noRJ5bA== Received: from mail-qk1-f169.google.com (mail-qk1-f169.google.com [209.85.222.169]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id 4QHwHN21nYzsqT for ; Fri, 12 May 2023 17:09:52 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-qk1-f169.google.com with SMTP id af79cd13be357-7576eb88a46so634150985a.3 for ; Fri, 12 May 2023 10:09:52 -0700 (PDT) X-Gm-Message-State: AC+VfDyIBnquMn36r+chYonopsNHTlF1HZAwrtZHMAN6/tbsZfu+LNtf Fr1r1eyePbu8JRJsD2YzrT+m4yLmDKTnR4lSWs4= X-Google-Smtp-Source: ACHHUZ4yU0zaXBHTVgOt1Hq9BzN/x87yR2kqprWIwC51nydiWnfGTf6pnoLP9NrgdOpWDTTCf36ewV821097Jnvcnrk= X-Received: by 2002:ad4:5cae:0:b0:61b:5dd6:1f26 with SMTP id q14-20020ad45cae000000b0061b5dd61f26mr38914280qvh.28.1683911391647; Fri, 12 May 2023 10:09:51 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Kyle Evans Date: Fri, 12 May 2023 12:09:40 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: enabling same PPI interrupt to all CPU in ARM64 SMP To: Souradeep Chakrabarti Cc: Wei Hu , "freebsd-hackers@FreeBSD.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-ThisMailContainsUnwantedMimeParts: N On Fri, May 12, 2023 at 9:51=E2=80=AFAM Souradeep Chakrabarti wrote: > > > > > >-----Original Message----- > >From: Souradeep Chakrabarti > >Sent: Monday, May 8, 2023 6:39 PM > >To: Kyle Evans > >Cc: Wei Hu ; freebsd-hackers@FreeBSD.org > >Subject: enabling same PPI interrupt to all CPU in ARM64 SMP > > > >Hi , > > > >While using SMP in ARM64 Hyper-V we are getting stuck in boot if there i= s a > >interrupt for VMBus coming to CPU1 and VMBus interrupt handler is not ge= tting > >that interrupt. > > > >In ARM64 Hyper-V we are using IRQ18 for VMBus and it is a PPI interrupt. > > > >But Hypev-V host sends interrupt to this IRQ 18 for both CPU0 and CPU1 i= n 2CPU > >system. > >This is based on the corresponding VMBus channel which assigned with the= CPU. > > > >Now VMBus ISR is getting the interrupt in CPU0 but not getting from CPU1= . > >Any idea, how we can use the same PPI 18 for all the CPU cores? > > > >Any help will be appreciated, as this is blocking the enablement of Free= BSD in Azure > >ARM64. > [Souradeep] > Can someone please help me it. > Looking at least at the GIC implementation, it looks like this is a known limitation: 875 /* 876 * XXX - In case that per CPU interrupt is going to be enabled in time 877 * when SMP is already started, we need some IPI call which 878 * enables it on others CPUs. Further, it's more complicated as 879 * pic_enable_source() and pic_disable_source() should act on 880 * per CPU basis only. Thus, it should be solved here somehow. 881 */ 882 if (isrc->isrc_flags & INTR_ISRCF_PPI) 883 CPU_SET(PCPU_GET(cpuid), &isrc->isrc_cpu); I think we need something /like/ this: https://people.freebsd.org/~kevans/ppi.diff, though it still has the caveat that PPIs effectively cannot be fully setup before SI_SUB_SMP. So, it's likely almost a NOP for existing platforms (will emit a warning with bootverbose for armv8 timers) but might do the trick for you. Thanks, Kyle Evans