Re: Cross compiling user applications for armv7

On Fri, Sep 12, 2025 at 2:11 PM Michał Kruszewski <mkru@protonmail.com>
wrote:

> > On a small scale, you can do a 'make buildenv' in a built FreeBSD src
> tree.
> > That will give you a shell you can build your kernel modules in that has
> all the env vars set up to get the right compiler, the right kernel tree,
> etc. this works well for quick debug cycles to copy the .ko to load /
> unload / debug on the target. There's some scripting features along with
> this so you can build your 'out of tree' modules with the FreeBSD tree as
> part of your build orchestration.
>
> Can I also build user space applications in that environment?
>

You can, and I have. However, it gets tricky if they have dependencies on
other things (a problem poudriere has solved). It's mostly documented in
build(7)


> > I can provide more details if need be.
> > I've been doing this for years...
>
> Please, do.
>

OK. I'll add it to my backlog. No, I'll update build(7) with (a) the
buildenvvars target (which gives you the vars you need) and the BUILDENV_SHELL
variable that can be leverages to do the builds (by setting it to 'sh -c
"cd /some/where && make all"').

Please see
https://cgit.freebsd.org/src/commit/?id=cbba9f7eeead2f5c64e178d668dd314df70d7d56
for the updated details.


> > On the larger scale, you can use poudriere.
> > It supports cross building ports with its 'bulk' command.
> > If you can wrap your applications inside of the ports world, then this
> works out well.
> > It uses qemu bsd-user to emulate the armv7 environment, but has special
> compilers it builds that run amd64 binaries to produce armv7 code by
> default so the building is usually fast, just the interpretation of
> binaries use to build would be slower.
>
> I tried.
> I overcome two problems.
> The third one overcome me.
> When I try to create poudriere jail with:
>   doas poudriere jail -c -j armv7 -v 14.3-R ELEASE -a armv7
> I get the following error:
>   [00:00:00] Cross-building ports for armv7 on amd64 requires QEMU
>   [00:00:00] Creating armv7 fs at /usr/local/poudriere/jails/armv7... done
>   [00:00:00] FREEBSD_HOST from config invalid; defaulting to
> https://download.FreeBSD.org
>   [00:00:00] Fetching MANIFEST for FreeBSD 14.3-RELEASE armv7
>   fetch:
> https://download.FreeBSD.org/releases/armv7/armv7/14.3-RELEASE/MANIFEST:
> Not Found
>   fetch:
> https://download.FreeBSD.org/releases/armv7/armv7/14.3-RELEASE/MANIFEST:
> Not Found
>   [00:00:01] Error: Failed to fetch from
> https://download.FreeBSD.org/releases/armv7/armv7/14.3-RELEASE/MANIFEST
>   [00:00:01] Error while creating jail, cleaning up.
>   [00:00:01] Removing armv7 jail... don
>

Yea. arm isn't packaged right, so you'll get that error, alas. I use the
build from source option when I do this.

Warner


> Regards,
> Michał Kruszewski
>
>
> Sent with Proton Mail <https://proton.me/mail/home> secure email.
>
> On Friday, September 12th, 2025 at 8:16 PM, Warner Losh <imp@bsdimp.com>
> wrote:
>
> OK.
>
> So there's a few things you can do.
>
> On a small scale, you can do a 'make buildenv' in a built FreeBSD src
> tree. That will give you a shell you can build your kernel modules in that
> has all the env vars set up to get the right compiler, the right kernel
> tree, etc. this works well for quick debug cycles to copy the .ko to load /
> unload / debug on the target. There's some scripting features along with
> this so you can build your 'out of tree' modules with the FreeBSD tree as
> part of your build orchestration. I can provide more details if need be.
> I've been doing this for years...
>
> On the larger scale, you can use poudriere. It supports cross building
> ports with its 'bulk' command. If you can wrap your applications inside of
> the ports world, then this works out well. It uses qemu bsd-user to emulate
> the armv7 environment, but has special compilers it builds that run amd64
> binaries to produce armv7 code by default so the building is usually fast,
> just the interpretation of binaries use to build would be slower.
>
> https://wiki.freebsd.org/Ports/BuildingPackagesThroughEmulation has some
> data, but it's a bit dated (mips and armv6 have aged out of support, for
> example).
>
> https://www.digitalocean.com/community/tutorials/how-to-set-up-a-poudriere-build-system-to-create-packages-for-your-freebsd-servers
> and
> https://phaq.phunsites.net/2015/10/11/freebsd-on-armv6-cross-compile-performance-optimization-for-poudriere/
> provide some good background and might be less data, though less targeted
> for what you want to do. Googling variations on 'poudriere cross build' may
> yield even more results.
>
> Warner
>
>
> On Fri, Sep 12, 2025 at 10:47 AM Michał Kruszewski <mkru@protonmail.com>
> wrote:
>
>> FreeBSD riis 14.3-RELEASE FreeBSD 14.3-RELEASE
>> releng/14.3-n271432-8c9ce319fef7 GENERIC amd64
>>
>>
>> Regards,
>> Michał Kruszewski
>>
>>
>> Sent with Proton Mail <https://proton.me/mail/home> secure email.
>>
>> On Friday, September 12th, 2025 at 6:40 PM, Warner Losh <imp@bsdimp.com>
>> wrote:
>>
>>
>>
>> On Fri, Sep 12, 2025 at 10:19 AM Michał Kruszewski <mkru@protonmail.com>
>> wrote:
>>
>>> Is there any tutorial on how to cross compile custom user application
>>> for armv7?
>>> I struggle t find any.
>>> Cross compiling the system is pretty easy.
>>> However, how can one cross compile custom user application or kernel
>>> drivers.
>>>
>>
>> What's your host?
>>
>> Warner
>>
>>
>>
>