LLDB on FreeBSD/arm64 and arm

Wed Feb 17 15:22:07 UTC 2021

Am 16.02.2021 um 11:01 schrieb Michal Meloun <meloun.michal at gmail.com <mailto:meloun.michal at gmail.com>>:
> On 16.02.2021 14:34, Dr. Rolf Jansen wrote:
>> Am 05.02.2021 um 13:40 schrieb Michał Górny <mgorny at gentoo.org>:
>>> On Fri, 2021-02-05 at 12:30 -0300, Dr. Rolf Jansen wrote:
>>>> Am 03.02.2021 um 14:53 schrieb Ed Maste <emaste at freebsd.org>:
>>>> 
>>>>> As you may know, the FreeBSD Foundation sponsored Moritz Systems to
>>>>> improve LLDB's support of non-x86 FreeBSD targets (and some other
>>>>> improvements applicable to all targets). This builds on their
>>>>> earlier
>>>>> work to update x86 support.
>>>>> 
>>>>> Initial changes for arm64 and arm support were recently committed
>>>>> upstream. Details are in these code reviews, if you're interested:
>>>>> 
>>>>> https://reviews.llvm.org/D95297 (arm64)
>>>>> https://reviews.llvm.org/D95696 (arm)
>>>>> 
>>>>> arm and arm64 users with an interest in LLDB are encouraged to build
>>>>> LLDB from the upstream git repository, try out the support, and
>>>>> report
>>>>> issues or positive results.
>>>>> 
>>>>> It should build using the regular build process as described at
>>>>> https://lldb.llvm.org/resources/build.html
>>>>> 
>>>>> At present all four of 32- and 64-bit arm and x86 use LLDB's
>>>>> non-legacy debug support by default. Legacy support will be removed
>>>>> from LLDB upon completion of Moritz' work (once all architectures
>>>>> are
>>>>> patched) and there is no need to try out legacy support.
>>>> 
>>>> I am interested to build LLDB from the upstream repository. As a
>>>> matter of fact a few years ago I did this already directly on a
>>>> BeagleBone Black, however in the meantime the LLVM sources had grown
>>>> beyond the capacity a BBB could compile. Already at that time a rather
>>>> limited build took more than 48 hs.
>>>> 
>>>> I set up an ARMv7 cross building environment for kernel and world on a
>>>> reasonably equipped i7 machine. Can I somehow use this for building
>>>> LLDB as well?
>>> 
>>> Yes.  See our summary for some tips:
>>> 
>>> https://www.moritz.systems/blog/freebsd-remote-process-plugin-on-non-x86-architectures/
>>> 
>>> Go for 'Cross-compiling LLVM'.  On ARMv7, you'll have to change
>>> the target and host triplets to armv7-unknown-freebsd13.0-gnueabihf
>>> and LLVM target to ARM.
>>> 
>>>> I provided ssh root access to one of my BBBs to Michał Górny for
>>>> testing his work, and he left a LLDB on the system, I don't know at
>>>> which stage, though. When debugging an advanced project of mine with
>>>> this one I see some show stoppers (as with the original LLDB).
>>>> Perhaps, I could be of help to improve LLDB.
>>> 
>>> If it's about watchpoints, then we need to support them on kernel level
>>> first.  For anything else, we need to figure out what's wrong/missing.
>> It took a while until I got it working - „too many variables, so few time“
>> In the toolchains file, I needed to add a definition for sysroot. When passing it with the C_FLAGS and CXX_FLAGS, I got typical 32 vs. 64 bit errors very early in the build process. Something like size_t is not long. The working toolchain file for ARMv7 is the following — <SYSROOT> is the placeholder for the actual path, here it is /root/install/BBB/sysroot:
>>   set(CMAKE_C_FLAGS "-target armv7-unknown-freebsd13.0-gnueabihf")
>>   set(CMAKE_CXX_FLAGS "-target armv7-unknown-freebsd13.0-gnueabihf")
>>   set(CMAKE_SYSROOT <SYSROOT>)
>>   set(CMAKE_SYSTEM_NAME "FreeBSD")
>>   set(CMAKE_FIND_ROOT_PATH <SYSROOT>)
>>   set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
>>   set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
>>   set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
>> Before, I found this out, I meandered between versions of LLVM (I compiled 10.x, 11.x, 12.x and HEAD), and I switched to another build machine, before it was FreeBSD 12.2-RELEASE on a real i7 and now I build in a VM running FreeBSD 13.0-BETA2. However, without the definition of CMAKE_SYSROOT all this did not work out.
>> Now I deployed a lldb to my BeagleBone running 13-BETA2 as well:
>> lldb version 13.0.0 (https://github.com/llvm/llvm-project.git revision b40fde062c306584d88243de11175187e754ce3b)
>>  clang revision b40fde062c306584d88243de11175187e754ce3b
>>  llvm revision b40fde062c306584d88243de11175187e754ce3b
>> I am able to debug simple programs, but the happiness ends already when trying to enter a subroutine. At least this worked well with lldb 11.0.1, which has been build by using in the course of world by WORLD_FLAGS="MK_LLDB=yes“. However both bail out, when I try to debug a bigger project of mine.
>> (lldb) target create "./CyDaemon"
>> Current executable set to '/root/install/CyDaemon/CyDaemon' (arm).
>> (lldb) settings set -- target.run-args  "-f" "-l" "80" "-s" "443"
>> (lldb) r
>> Process 767 launched: '/root/install/CyDaemon/CyDaemon' (arm)
>> Process 767 stopped
>> * thread #1, name = 'CyDaemon', stop reason = signal SIGILL: illegal instruction
>>    frame #0: 0x20290048
>> ->  0x20290048: vmull.p64 q0, d0, d0
>>    0x2029004c: bx     lr
>>    0x20290050: ldr    r0, [pc, #0x68]
>>    0x20290054: add    r1, pc, #100
>> (lldb)
>> The question now is, how to continue from here? Do we continue? If yes, do we build on FreeBSD 13.0-RELEASE (BETAx for some more days) or do we build on 14-CURRENT? Which version of lldb do we debug? A release version or HEAD?
>> The last lldb which worked absolutely well on the BBB was 3.9 or was it 4.x. I know, that I never got 5.x to compile on the BBB. However now, that my cross-building environment has been set up, I probably will check that as well.
>> Best regards
>> Rolf
> 
> Hi Rolf,
> This is quite normal behavior.. Your program uses crypt library ant this library determines presence of crypto extension instructions (Cortex A9 have not these). The library have SIGILL signal handler installed and if vmull instruction is not supported it switches to pure sw mode. In this case do nothing else press continue...

OK, this make sense. Actually, I need to continue 5 times. Now I pass a startup script file on the lldb command line so it skips this annoyance automatically:

-s lldbstartup

  run
  process handle -n false -p true -s false SIGILL
  continue

I now compiled release/11.x and this works well, except the longstanding issue, by which lldb in the GUI loses the focus when stepping through a threaded program — see https://lists.llvm.org/pipermail/lldb-dev/2017-November/012954.html <https://lists.llvm.org/pipermail/lldb-dev/2017-November/012954.html>

Anyhow, this thread is about testing new features which made it into LLDB for FreeBSD arm[64|], and my question remains, where did these additions landed (main?, release/12.x?, release/11.x?) and what would I test?

Best regards

Rolf