arm64 as Tier 1 for FreeBSD 13

[Ed Maste]

The FreeBSD core team recently modernized and updated the Platform
Tier documentation, available at
https://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/archs.html.

I believe that arm64 as a platform is close to being Tier 1 and would
like to determine what's still needed to get there. Many of the Tier 1
guarantees are already provided on arm64. and I won't copy them here.

I've provided my comments on what I see as the gaps between the Tier 1
attributes and arm64's current state, and am very interested in
hearing about anything I've missed.

> Binary updates and source patches for Security Advisories and Errata
> Notices will be provided for supported releases.

We don't do this today, but have the ability to do so for arm64 server
platforms. (Due to its design, freebsd-update does not work
particularly well on devices with slow root filesystems such as SD
cards.)

> Changes to userland ABIs will generally include compatibility shims to
> ensure correct operation of binaries compiled against any stable branch
> where the platform is Tier 1.  These shims might not be enabled in the
> default install. If compatibility shims are not provided for an ABI
> change, the lack of shims will be clearly documented in the release
> notes.

In the past we've used the fact that a platform is no Tier 1 to make
ABI-breaking changes, like switching time_t from 32 to 64 bits. I see
no issue guaranteeing we will not do that on arm64.

> Changes to certain portions of the kernel ABI will include compatibility
> shims to ensure correct operation of kernel modules compiled against the
> oldest supported release on the branch. Note that not all parts of the
> kernel ABI are protected.

No concern here either - in practice I believe most of the issues that
could arise here will be machine-independent and need to be addressed
for all platforms.

> Official binary packages for third party software will be provided by the
> ports team. For embedded architectures, these packages may be cross-built
> from a different architecture.

We do this today, although on somewhat slow and unstable hardware. I
expect faster arm64 package build machines to be added in the near
future.

> New features which are not inherently platform-specific will be fully
> functional on all Tier 1 architectures.

This introduces some additional commitments on those developing new
features, but in practice I believe we already generally expect new
functionality to work on arm64.

> Tier 1 platforms should be fully documented. Basic operations will be
> documented in the FreeBSD Handbook.
Some Handbook updates are warranted for arm64, although this is true

for existing Tier-1 architectures as well (e.g. documenting amd64 BIOS
booting but not UEFI).

> Build and test automation support either in the FreeBSD.org cluster or
> some other location easily available for all developers.  Embedded
> platforms may substitute an emulator available in the FreeBSD.org cluster
> for actual hardware.

We build FreeBSD/arm64 in CI and smoke test on physical hardware
today. More is needed (including adding a capable ref machine to the
cluster) but I don't see any significant issues here.

> Developers should be able to build packages on commonly available,
> non-embedded Tier 1 systems. This can mean either native builds if
> non-embedded systems are commonly available for the platform in question,
> or it can mean cross-builds hosted on some other Tier 1 architecture.

This is somewhat of a challenge today - there aren't many arm64
platforms readily available in a configuration most suited to
developer use, such as a 4- or 8-core system with 16GB of RAM and
SATA- or NVMe-connected storage. Smaller systems (e.g. Pine64) are
readily available but not quite capable enough; larger systems (e.g.
Marvell ThunderX and Ampere eMAG) are out of reach for typical
developer use. User-mode QEMU cross-builds are a possibility, but this
item is one that should resolve over time as new platforms become
available.