CTF vs dwarf4

[Pedro Giffuni]

Hi Robert;

On 11.01.2014 15:59, Robert Mustacchi wrote:
> Sorry for the delay in getting back to you.

Don't worry, I am doing other stuff myself.
> On 01/09/2014 12:44 PM, Pedro Giffuni wrote:
>>> Currently I've been retooling ctfconvert + ctfmerge to be actually
>>> implemented in terms of libctf to make it easier for other things to
>>> actually consume, such as more long term illumos ld(1) and ideally the
>>> compilers themselves. None of this code is finished or in illumos yet.
>>> That really means that I have a slightly more useful merge (and as a
>>> result diff) written, and convert is kind of a work in progress.
>> As a side note ... completely unrelated to the issue at hand.
>>
>> I don't know much about how DTrace uses dwarf, but I had some contact
>> with the Oracle guys doing the linux port and I think they are doing
>> something drastically different to illumos (and FreeBSD). They are
>> avoiding the issue of getting DTrace in the kernel and they are using
>> dwarf2 instead of the stabs-like format supported by CTF. Their stuff is
>> in a GIT repository somewhere but I would have to dig deep in my private
>> email for the finer details. They were likely to have problems with
>> dwarf4 too.
> Today DTrace uses this kind of information so that it knows how to refer
> to types in both userland (on illumos) and the kernel, and how if you do
> args[x], it knows the type and allows you to dereference it.
>
>>> Probably the simplest way forward that we can all leverage is writing a
>>> new ctfconvert in terms of DWARF4 that can exist side by side the old
>>> one. eg. if you have an older compiler with dwarf2 (or you're still
>>> stuck with stabs or some other language has its own exquisite format),
>>> that'll still work. illumos last updated libdwarf in 2011. If there's a
>>> newer libdwarf that has better DWARF4 support or we need to come up with
>>> something else, let's make sure not to duplicate effort.
>> I recall Illumos is using SGI's libdwarf which we have avoiding using
>> mainly because it's copyleft but I recall there were also some
>> implementation details that made it undesirable. I haven't followed the
>> SGI libdwarf but when I last tried to update the pre-packaged port I
>> noticed it  was becoming more difficult to port to non-linux (that may
>> have been solved though).
>>
>> Some FreeBSD developers reimplemented libelf and libdwarf specifically
>> to use it in DTrace:
>> http://sourceforge.net/p/elftoolchain/wiki/libdwarf/
>>
>> Their implementation is very clean and portable and it's basically what
>> we have been using but we have to update it and I am not sure about
>> their dwarf4 support.
>>
>> If illumos and freebsd can use the same libdwarf than that would be
>> good. OTOH, with llvm having support for dwarf4 it would certainly be
>> tempting to try to integrate what we can with the toolchain. I don't see
>> illumos advancing much in the llvm direction though, so this option
>> would be good for FreeBSD but probably not for illumos.
> I don't see us moving to the llvm/clang world anytime soon unless
> someone else picks that up and starts teaching it some of the things
> we've taught gcc.
I read somewhere that Oracle had been sponsoring someone to work on the 
Solaris clang support, so I couldn't call off the chances.

I looked at it briefly (I am not really working on this atm), and one 
small inconvenience is that the llvm stuff is C++.

> But as we slowly move compilers, we're also probably
> going to stick to forcing it to emit dwarf2 for the core gate, unless we
> have a compelling reason to switch to dwarf4. Though despite that, for
> at least Joyent's equivalent of ports (netbsd pkgsrc), it's probably
> more worthwhile to get the dwarf4 support.

I think it's mostly unavoidable. You just can't keep using an old 
unsupported compiler for too long.

> As for libdwarf, the ctf tools are currently our only consumer of
> libdwarf. There isn't anything that compels us to stick with it, so we
> could probably move towards the same libdwarf that FreeBSD works on.
> I'll take a look at that in my current CTF workspace. So if FBSD is
> willing to continue down the libdwarf angle, I'd be happy to collaborate
> where I can with that.

Luckily both the SGI and FreeBSD/elftoolchain libdwarf use the same API.
I haven't had much luck finding out about the dwarf4 support in it but I 
think it looks like the best option.

Pedro.