Re: Cross compiling user applications for armv7

On Sat, Sep 13, 2025 at 10:45 AM Michał Kruszewski <mkru@protonmail.com>
wrote:

> Ok, let's start from the beginning because I feel a little bit lost.
> The buildenv concept is ok when you have no dependencies.
> However, I have.
>
> I see 3 potential options.
>   1. Use arm64 as an intermediate platform.
>      This is unacceptable approach for me.
>   2. Setup poudriere for armv7 manually.
>      I tried for 2 days and simply failed.
>   3. Setup qemu, and install dependencies and build custom code inside VM.
>
> Solutions 2 and 3 are acceptable for me.
> I guess 2 is better than 3, as it should be more performant.
> However, I have no idea how to implement idea 2 or 3 in practice.
>
> I also don't understand why setting up poudriere for armv7 is so hard.
> What is missing by default that I can't do it with a single command?
>

You can do it with a single command, just not from binaries because our
armv7 packaging was always just for embedded, so we didn't do the right
thing for normal installs.

However,
https://forums.freebsd.org/threads/poudriere-image-creation-for-arm-architecture.91870/
has a good recipe for the source build. Have you tried it.

poudriere jail -c -x -j armv7 -m git+https -a arm.armv7 -v releng/14.3 -K
GENERIC
poudriere ports -c -m git+https -B main -p HEAD

are the commands to setup the root and to get ports so you can do a
poudriere bulk to build the ports you are interested in.

Why the things that are missing are so hard to recreate manually?
>

So to do #2 manually... The following blog, something I wrote long ago, but
that should still work
https://wiki.freebsd.org/WarnerLosh/A_Brief_Guide_To_Cross_Compiling_FreeBSD
has the steps (though you'll need to s/armeb/armv7/ in it). There's three
commands to creating an image. But the results are almost the same as the
above the first poudriere command while being less flexible or performant.

As for #3, it's a bit less performant, but you can run FreeBSD inside of
qemu using the virt* devices easily enough (though I think there's a bug
with 14.3's kernel so you have to use em NIC not virtnet):

https://bsdimp.blogspot.com/2023/12/freebsdarmv7-in-qemu.html

has a writeup I did a while ago. It's still accurate and suggests the
virtnet bug may be fixed in all 14.x releases. I run this setup from time
to time still to test things in a armv7 system. It's OK if what you are
building is small, but can get tedious if you have lots of complex
dependencies.

Hope this helps.

Warner


> Regards,
> Michał Kruszewski
>
>
> Sent with Proton Mail <https://proton.me/mail/home> secure email.
>
> On Saturday, September 13th, 2025 at 6:09 PM, Warner Losh <imp@bsdimp.com>
> wrote:
>
>
>
> On Sat, Sep 13, 2025 at 9:50 AM Michał Kruszewski <mkru@protonmail.com>
> wrote:
>
>> It is more about how hard it is to setup an environment for cross
>> compiling your own kernel modules or applications for armv7.
>> I don't really care whether I use Buildroot way of doing things or
>> FreeBSD way.
>> The difference is that with Buildroot it takes 10 minutes to setup such
>> environment.
>> With FreeBSD I am struggling for two days.
>> And the only advice I get is to use arm64 machine.
>> Well, I don't have any.
>> And even if I had, I wouldn't use it, because I find this concept deeply
>> "misengineered".
>>
>
> And it is bad engineering. Mark sometimes gives bad advice that's based on
> his experience where it was "easier" to do it this way. I just wish I'd
> replied more quickly.
>
>> I know that you want to help, and I appreciate it.
>> I just feel frustrated with the complexity of this conceptually trivial
>> task.
>>
>
> No. I get it. I'm frustrated too since I've been working for years to make
> it better, and there's still these rough edges that turn people off.
>
> Warner
>
>> Regards,
>> Michał Kruszewski
>>
>>
>> Sent with Proton Mail secure email.
>>
>> On Saturday, September 13th, 2025 at 4:29 PM, Mark Millard <
>> marklmi@yahoo.com> wrote:
>>
>> >
>> > On Sep 13, 2025, at 00:20, Michał Kruszewski mkru@protonmail.com wrote:
>> >
>> > > Basically you are saying that I should have a separate physical arm64
>> machine for doing embedded armv7 development on FreeBSD.
>> > > If I understand this correctly, I just can't understand how can
>> people say that FreeBSD is embedded friendly.
>> > > This is so much complex and convoluted compared to just using, for
>> example, Buildroot on Linux.
>> > > This also doesn't feel like a *nix way of solving things.
>> > > I thought that after cross compiling world and kernel, there is some
>> shell script that I can simply source to start cross-development for target
>> platform.
>> >
>> >
>> > I found this AI based note about Buildroot's way of
>> > putting cross-build toolchains to use:
>> >
>> > QUOTE
>> > Buildroot pre-configures and compiles all necessarily
>> > packages during the build process to create a custom
>> > root filesystem.
>> > . . .
>> > Buildroot focuses on static builds, meaing all selected
>> > packages are compiled and integrated into the final root
>> > filesystem image during the build process. There is no
>> > concept of installing or updating packages on the target
>> > after the initial build.
>> > END QUOTE
>> >
>> > This is certainly not what the FreeBSD ports tree is
>> > set up for. Buildroot or analogous does not exist for
>> > FreeBSD --or most *BSD or Unix-- to my knowledge.
>> >
>> > May be NetBSD's pkgsrc cross build support:
>> >
>> >
>> https://cdn.netbsd.org/pub/pkgsrc/current/pkgsrc/doc/HOWTO-use-crosscompile
>> >
>> > would be closer to how you like to work? (I've never
>> > used such.)
>> >
>> > > Regards,
>> > > Michał Kruszewski
>> > >
>> > > Sent with Proton Mail secure email.
>> > >
>> > > On Saturday, September 13th, 2025 at 12:23 AM, Mark Millard
>> marklmi@yahoo.com wrote:
>> > >
>> > > > On Sep 12, 2025, at 14:10, Michał Kruszewski mkru@protonmail.com
>> wrote:
>> > > >
>> > > > > A simple way is to set up an armv7 chroot / jail on an arm64 host
>> such as a Raspberry Pi (Apple devices don't work!).
>> > > > > Then it's just like a native environment, but usually much faster.
>> > > >
>> > > > > I don't have physical arm64 host.
>> > > > > I also don't want to create a VM.
>> > > > > The whole idea seems weird and unnatural.
>> > > >
>> > > > Ignoring the RPi4B detail:
>> > > >
>> > > > FreeBSD official armv7 port-packages are built this way
>> > > > on arm64 hardware that natively supports user-space
>> > > > armv7 code. (ampere* systems are used.)
>> > > >
>> > > > FreeBSD official i386 port-packages are built this way
>> > > > on amd64 hardware that natively supports user-space
>> > > > i386 code.
>> > > >
>> > > > No use of qemu variants of any kind: no attempt at
>> > > > non-native-capable host environments.
>> > > >
>> > > > (Back when amd64 and qemu was used for the likes
>> > > > of armv7, lots of stuff failed to build mcuh of
>> > > > the time over the years --stuff that builds
>> > > > just fine now. This was abandoned after native
>> > > > became available. armv6 was always qemu based
>> > > > on amd64 and could not build much as of the last
>> > > > time a build was run.)
>> > > >
>> > > > > It sounds more like an exotic workaround.
>> > > >
>> > > > It is the official technique used for what FreeBSD
>> > > > distributes for armv7 port-packages.
>> > > >
>> > > > > Regards,
>> > > > > Michał Kruszewski
>> > > > >
>> > > > > Sent with Proton Mail secure email.
>> > > > >
>> > > > > On Friday, September 12th, 2025 at 10:46 PM, Robert Clausecker
>> fuz@fuz.su wrote:
>> > > > >
>> > > > > > Hi Michał,
>> > > > > >
>> > > > > > A simple way is to set up an armv7 chroot / jail on an arm64
>> host such as
>> > > > > > a Raspberry Pi (Apple devices don't work!). Then it's just like
>> a native
>> > > > > > environment, but usually much faster.
>> > > > > >
>> > > > > > Yours,
>> > > > > > Robert Clausecker
>> > > > > >
>> > > > > > Am Fri, Sep 12, 2025 at 04:19:42PM +0000 schrieb Michał
>> Kruszewski:
>> > > > > >
>> > > > > > > Is there any tutorial on how to cross compile custom user
>> application for armv7?
>> > > > > > > I struggle t find any.
>> > > > > > > Cross compiling the system is pretty easy.
>> > > > > > > However, how can one cross compile custom user application or
>> kernel drivers.
>> > > > > > >
>> > > > > > > Regards,
>> > > > > > > Michał Kruszewski
>> > > > > > >
>> > > > > > > Sent with Proton Mail secure email.
>> > > > > >
>> > > > > > --
>> > > > > > () ascii ribbon campaign - for an encoding-agnostic world
>> > > > > > /\ - against html email - against proprietary attachments
>> > > >
>> > > > ===
>> > > > Mark Millard
>> > > > marklmi at yahoo.com
>> >
>> >
>> > ===
>> > Mark Millard
>> > marklmi at yahoo.com
>>
>>
>