Re: Cross compiling user applications for armv7

On Sep 12, 2025, at 13:11, Michał Kruszewski <mkru@protonmail.com> wrote:

> > On a small scale, you can do a 'make buildenv' in a built FreeBSD src tree.
> > That will give you a shell you can build your kernel modules in that has all the env vars set up to get the right compiler, the right kernel tree, etc. this works well for quick debug cycles to copy the .ko to load / unload / debug on the target. There's some scripting features along with this so you can build your 'out of tree' modules with the FreeBSD tree as part of your build orchestration.
> 
> Can I also build user space applications in that environment?
> 
> > I can provide more details if need be.
> > I've been doing this for years...
> 
> Please, do.
> 
> > On the larger scale, you can use poudriere.

Not easily for armv7, at least not for some typical
ways of attempting the jail creation. (See later
notes.)

> > It supports cross building ports with its 'bulk' command.
> > If you can wrap your applications inside of the ports world, then this works out well.
> > It uses qemu bsd-user to emulate the armv7 environment, but has special compilers it builds that run amd64 binaries to produce armv7 code by default so the building is usually fast, just the interpretation of binaries use to build would be slower.

"special compilers" are for context that use the
system compilers. If a lang/gcc* or a devel/llvm*
is used, for example, it does not get native-based
cross compile performance.

Such lang/gcc* or a devel/llvm* usage is likely
faster on the likes of an RPi4B working natively
on a armv7 jail. (There is a working way to set
up such a poudriere armv7 jail up on such a system
that natively supports armv7 user-space code.)

(There are a couple of EDK2 drafts around that
allow some RPi5B's to be used for such things
with USB media or NVMe media.)

> I tried.
> I overcome two problems.
> The third one overcome me.
> When I try to create poudriere jail with:
>   doas poudriere jail -c -j armv7 -v 14.3-R ELEASE -a armv7
> I get the following error:
>   [00:00:00] Cross-building ports for armv7 on amd64 requires QEMU
>   [00:00:00] Creating armv7 fs at /usr/local/poudriere/jails/armv7... done
>   [00:00:00] FREEBSD_HOST from config invalid; defaulting to https://download.FreeBSD.org
>   [00:00:00] Fetching MANIFEST for FreeBSD 14.3-RELEASE armv7
>   fetch: https://download.FreeBSD.org/releases/armv7/armv7/14.3-RELEASE/MANIFEST: Not Found
>   fetch: https://download.FreeBSD.org/releases/armv7/armv7/14.3-RELEASE/MANIFEST: Not Found
>   [00:00:01] Error: Failed to fetch from https://download.FreeBSD.org/releases/armv7/armv7/14.3-RELEASE/MANIFEST
>   [00:00:01] Error while creating jail, cleaning up.
>   [00:00:01] Removing armv7 jail... done

There is a path:

https://download.freebsd.org/releases/arm/armv7/

( note, not armv7/armv7/ ), but it only directly contains:

https://download.freebsd.org/releases/arm/armv7/ISO-IMAGES/


poudriere is not set up to deal with:

https://download.freebsd.org/releases/arm/armv7/ISO-IMAGES/14.3/FreeBSD-14.3-RELEASE-arm-armv7-GENERICSD.img.xz

which is the only type of armv7 image or such present
--and which has no manifest.

A different way of creating the armv7 jail's world would
need to be used. (I've not tried such alternatives for
non-native capable armv7 environments in very many years.)

> Regards,
> Michał Kruszewski
> 
> 
> Sent with Proton Mail secure email. 
> 
> On Friday, September 12th, 2025 at 8:16 PM, Warner Losh <imp@bsdimp.com> wrote:
>> OK. 
>> 
>> So there's a few things you can do.
>> 
>> On a small scale, you can do a 'make buildenv' in a built FreeBSD src tree. That will give you a shell you can build your kernel modules in that has all the env vars set up to get the right compiler, the right kernel tree, etc. this works well for quick debug cycles to copy the .ko to load / unload / debug on the target. There's some scripting features along with this so you can build your 'out of tree' modules with the FreeBSD tree as part of your build orchestration. I can provide more details if need be. I've been doing this for years...
>> 
>> On the larger scale, you can use poudriere. It supports cross building ports with its 'bulk' command. If you can wrap your applications inside of the ports world, then this works out well. It uses qemu bsd-user to emulate the armv7 environment, but has special compilers it builds that run amd64 binaries to produce armv7 code by default so the building is usually fast, just the interpretation of binaries use to build would be slower.
>> 
>> https://wiki.freebsd.org/Ports/BuildingPackagesThroughEmulation has some data, but it's a bit dated (mips and armv6 have aged out of support, for example). 
>> https://www.digitalocean.com/community/tutorials/how-to-set-up-a-poudriere-build-system-to-create-packages-for-your-freebsd-servers and https://phaq.phunsites.net/2015/10/11/freebsd-on-armv6-cross-compile-performance-optimization-for-poudriere/ provide some good background and might be less data, though less targeted for what you want to do. Googling variations on 'poudriere cross build' may yield even more results.
>> 
>> Warner
>> 
>> 
>> On Fri, Sep 12, 2025 at 10:47 AM Michał Kruszewski <mkru@protonmail.com> wrote:
>> FreeBSD riis 14.3-RELEASE FreeBSD 14.3-RELEASE releng/14.3-n271432-8c9ce319fef7 GENERIC amd64
>> 
>> 
>> Regards,
>> Michał Kruszewski
>> 
>> 
>> Sent with Proton Mail secure email. 
>> 
>> On Friday, September 12th, 2025 at 6:40 PM, Warner Losh <imp@bsdimp.com> wrote:
>>> 
>>> 
>>> On Fri, Sep 12, 2025 at 10:19 AM Michał Kruszewski <mkru@protonmail.com> wrote:
>>> Is there any tutorial on how to cross compile custom user application for armv7?
>>> I struggle t find any.
>>> Cross compiling the system is pretty easy.
>>> However, how can one cross compile custom user application or kernel drivers.
>>> 
>>> What's your host?
>>> 
>>> Warner
>> 


===
Mark Millard
marklmi at yahoo.com