LLDB on FreeBSD/arm64 and arm

Tue Feb 16 13:34:44 UTC 2021

> Am 05.02.2021 um 13:40 schrieb Michał Górny <mgorny at gentoo.org>:
>> On Fri, 2021-02-05 at 12:30 -0300, Dr. Rolf Jansen wrote:
>>> Am 03.02.2021 um 14:53 schrieb Ed Maste <emaste at freebsd.org>:
>>> 
>>> As you may know, the FreeBSD Foundation sponsored Moritz Systems to
>>> improve LLDB's support of non-x86 FreeBSD targets (and some other
>>> improvements applicable to all targets). This builds on their
>>> earlier
>>> work to update x86 support.
>>> 
>>> Initial changes for arm64 and arm support were recently committed
>>> upstream. Details are in these code reviews, if you're interested:
>>> 
>>> https://reviews.llvm.org/D95297 (arm64)
>>> https://reviews.llvm.org/D95696 (arm)
>>> 
>>> arm and arm64 users with an interest in LLDB are encouraged to build
>>> LLDB from the upstream git repository, try out the support, and
>>> report
>>> issues or positive results.
>>> 
>>> It should build using the regular build process as described at
>>> https://lldb.llvm.org/resources/build.html
>>> 
>>> At present all four of 32- and 64-bit arm and x86 use LLDB's
>>> non-legacy debug support by default. Legacy support will be removed
>>> from LLDB upon completion of Moritz' work (once all architectures
>>> are
>>> patched) and there is no need to try out legacy support.
>> 
>> I am interested to build LLDB from the upstream repository. As a
>> matter of fact a few years ago I did this already directly on a
>> BeagleBone Black, however in the meantime the LLVM sources had grown
>> beyond the capacity a BBB could compile. Already at that time a rather
>> limited build took more than 48 hs.
>> 
>> I set up an ARMv7 cross building environment for kernel and world on a
>> reasonably equipped i7 machine. Can I somehow use this for building
>> LLDB as well?
> 
> Yes.  See our summary for some tips:
> 
> https://www.moritz.systems/blog/freebsd-remote-process-plugin-on-non-x86-architectures/
> 
> Go for 'Cross-compiling LLVM'.  On ARMv7, you'll have to change
> the target and host triplets to armv7-unknown-freebsd13.0-gnueabihf
> and LLVM target to ARM.
> 
>> I provided ssh root access to one of my BBBs to Michał Górny for
>> testing his work, and he left a LLDB on the system, I don't know at
>> which stage, though. When debugging an advanced project of mine with
>> this one I see some show stoppers (as with the original LLDB).
>> Perhaps, I could be of help to improve LLDB.
> 
> If it's about watchpoints, then we need to support them on kernel level
> first.  For anything else, we need to figure out what's wrong/missing.

It took a while until I got it working - „too many variables, so few time“

In the toolchains file, I needed to add a definition for sysroot. When passing it with the C_FLAGS and CXX_FLAGS, I got typical 32 vs. 64 bit errors very early in the build process. Something like size_t is not long. The working toolchain file for ARMv7 is the following — <SYSROOT> is the placeholder for the actual path, here it is /root/install/BBB/sysroot:

   set(CMAKE_C_FLAGS "-target armv7-unknown-freebsd13.0-gnueabihf")
   set(CMAKE_CXX_FLAGS "-target armv7-unknown-freebsd13.0-gnueabihf")
   set(CMAKE_SYSROOT <SYSROOT>)
   set(CMAKE_SYSTEM_NAME "FreeBSD")
   set(CMAKE_FIND_ROOT_PATH <SYSROOT>)
   set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
   set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
   set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

Before, I found this out, I meandered between versions of LLVM (I compiled 10.x, 11.x, 12.x and HEAD), and I switched to another build machine, before it was FreeBSD 12.2-RELEASE on a real i7 and now I build in a VM running FreeBSD 13.0-BETA2. However, without the definition of CMAKE_SYSROOT all this did not work out.

Now I deployed a lldb to my BeagleBone running 13-BETA2 as well:

lldb version 13.0.0 (https://github.com/llvm/llvm-project.git revision b40fde062c306584d88243de11175187e754ce3b)
  clang revision b40fde062c306584d88243de11175187e754ce3b
  llvm revision b40fde062c306584d88243de11175187e754ce3b

I am able to debug simple programs, but the happiness ends already when trying to enter a subroutine. At least this worked well with lldb 11.0.1, which has been build by using in the course of world by WORLD_FLAGS="MK_LLDB=yes“. However both bail out, when I try to debug a bigger project of mine.

(lldb) target create "./CyDaemon"
Current executable set to '/root/install/CyDaemon/CyDaemon' (arm).
(lldb) settings set -- target.run-args  "-f" "-l" "80" "-s" "443"
(lldb) r
Process 767 launched: '/root/install/CyDaemon/CyDaemon' (arm)
Process 767 stopped
* thread #1, name = 'CyDaemon', stop reason = signal SIGILL: illegal instruction
    frame #0: 0x20290048
->  0x20290048: vmull.p64 q0, d0, d0
    0x2029004c: bx     lr
    0x20290050: ldr    r0, [pc, #0x68]
    0x20290054: add    r1, pc, #100
(lldb)

The question now is, how to continue from here? Do we continue? If yes, do we build on FreeBSD 13.0-RELEASE (BETAx for some more days) or do we build on 14-CURRENT? Which version of lldb do we debug? A release version or HEAD?

The last lldb which worked absolutely well on the BBB was 3.9 or was it 4.x. I know, that I never got 5.x to compile on the BBB. However now, that my cross-building environment has been set up, I probably will check that as well.

Best regards

Rolf