From nobody Tue Nov 08 00:50:14 2022 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4N5qJf0QYmz4gsnv for ; Tue, 8 Nov 2022 00:50:26 +0000 (UTC) (envelope-from ehem@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "m5p.com", Issuer "R3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4N5qJd0nbNz3P46 for ; Tue, 8 Nov 2022 00:50:25 +0000 (UTC) (envelope-from ehem@m5p.com) Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7]) by mailhost.m5p.com (8.16.1/8.15.2) with ESMTPS id 2A80oENI002120 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Mon, 7 Nov 2022 19:50:20 -0500 (EST) (envelope-from ehem@m5p.com) Received: (from ehem@localhost) by m5p.com (8.16.1/8.15.2/Submit) id 2A80oEqI002119 for freebsd-arm@freebsd.org; Mon, 7 Nov 2022 16:50:14 -0800 (PST) (envelope-from ehem) Date: Mon, 7 Nov 2022 16:50:14 -0800 From: Elliott Mitchell To: freebsd-arm@freebsd.org Subject: FreeBSD/ARM64 on Xen/ARM64 Message-ID: List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-0.0 required=10.0 tests=KHOP_HELO_FCRDNS, T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on mattapan.m5p.com X-Rspamd-Queue-Id: 4N5qJd0nbNz3P46 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of ehem@m5p.com designates 74.104.188.4 as permitted sender) smtp.mailfrom=ehem@m5p.com X-Spamd-Result: default: False [-3.24 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.979]; NEURAL_HAM_MEDIUM(-0.96)[-0.963]; R_SPF_ALLOW(-0.20)[+a:c]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[m5p.com]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; ARC_NA(0.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; ASN(0.00)[asn:701, ipnet:74.104.0.0/16, country:US]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; TAGGED_FROM(0.00)[freebsd] X-ThisMailContainsUnwantedMimeParts: N Personally, I think merely being able to boot FreeBSD and Linux on a Raspberry PI 4B is boring. What I want is to boot both FreeBSD and Linux on a Raspberry PI 4B *at the same time*. I've got it operational, though it has only received light testing so far. Since it is the way to present things: https://gitlab.com/ehem/freebsd-src.git The branches of note are "main", "submit", "phase1" and "phase2". There are some other branches distantly related to Xen. Also note those branches are different spots in a series, "submit" contains everything. "main" is things which have presently been okayed in e-mail, but await some testing on x86 by Roger Pau Monné. A key item here is an interface point for non-x86 interrupt implementations has been added. With this no more changes to common code should be needed. "phase1" is a child of main. This is 4 commits which are approved, but 3 may have committers and the 4th may become unnecessary. "phase2" is a child of phase1. This is roughly the approach Julien Grall used for his proof of concept in 2015. Rather a lot of work has been done to clean things up and update to match the current FreeBSD. All of this needs approval, but it has been tested and it known to work (for me, at least). "submit" is a child of phase2. This is a batch attempting to go after some outstanding issues. The situation for these is interesting, they may be needed for context. One outstanding issue is where/when to probe for Xen. The presently tested approach is to probe during the console probe. The reason is this is the first place where Xen's presence/absence needs to be known (otherwise no console is available). Roger Pau Monné dislikes the approach, but is okay with it. As such I would need suggestions for alternative probe places. The constraints are, this needs to be BEFORE the console probe, this needs to be AFTER passed-in device-trees have been parsed. When Julien Grall did his proof of concept he implemented it on top of the kernel event interface. This seems reasonable for 2015, but there seems to be a chorus insisting upon reimplementing on top of INTRNG. I've been taking some looks at INTRNG and I think I've finally run into the documentation /I/ needed. There is a major problem with INTRNG though which needs fixing. If reimplement on top of INTRNG, the straightforward implementation of xen_arch_intr_release() would be to call intr_isrc_deregister(). If you examine the source of intr_isrc_deregister() there is a major issue: xen_arch_intr_release() -> intr_isrc_deregister() for non-IPI interrupts (common) -> isrc_release_counters() -> panic("%s: not implemented", __func__) Actual hardware hot-plug interrupt controllers are pretty uncommon due to their price. For interrupt controllers implemented as software, hot-plug is a cheap feature. Xen's event channel is a software interrupt implementation and interrupts being added or removed is routine. As such this is an absolute blocker. I had been taking a look at fixing isrc_release_counters(), but that is troublesome. Any approach is going to involve substantial work in several places. A different approach came to mind while looking at this situation though. isrc_release_counters() is the inverse of isrc_setup_counters(). If you look, isrc_setup_counters() is actually pretty gross. It is storing equivalent values into isrc_index and isrc_count (they are readily derived from each other). When you look further, there is even more ugliness. intrcnt_updatename() simply copies data from ->isrc_event to intrnames. Why is a redundant copy of this data needed? When you look at the other FreeBSD architectures near-identical ugliness is present. Always copying data to intrnames, then having duplicate fields on the interrupt structures. Well, "intrnames" and "intrcnt" were added in 2004. As such due to being around a long time there would be dozens, likely hundreds or even thousands of uses if it was a crucial feature. When I looked, what did I find? In the FreeBSD kernel "intrnames" and "intrcnt" are used in a grand total of 2 places. First, sys/kern/kern_intr.c uses them for the "hw.intrnames" and "hw.intrcnt" sysctl()s. Second, sys/kern/kern_clock.c uses them for the watchdog_fire() function. After 18 years there are a grand total of 2 uses. Removing the uses resulted in a net removal of more than 200 lines of source code. This has the characteristics of an albatross, not a valuable feature. Likely looked good in 2004, but now... As such a blocking feature for use of INTRNG is D36610. Is there a reviewer in the house? I'm guessing the initial reimplementation of watchdog_fire() will need a different approach, but I'm wondering what reviewers would prefer. D36610 is the large batch near the tip of the "submit" branch. Anything else for the portion between "phase1" and "phase2"? -- (\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) \BS ( | ehem+sigmsg@m5p.com PGP 87145445 | ) / \_CS\ | _____ -O #include O- _____ | / _/ 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445