arm64: userspace broken with jemalloc 4.0.0

[Andrew Turner]

On Tue, 1 Sep 2015 08:25:00 +0100
Julien Grall <julien.grall at citrix.com> wrote:

> Hi Andrew,
> 
> On 31/08/2015 22:06, Andrew Turner wrote:
> > On Sat, 29 Aug 2015 23:05:52 +0100
> > Julien Grall <julien.grall at citrix.com> wrote:
> >> I've built the latest freebsd master (r287263) for arm64 today.
> >> While trying to use the userspace I hit some ASSERT in jemalloc:
> >>
> >> # ls
> >> <jemalloc>: /usr/src/freebsd/lib/libc/../../contrib/jemalloc/include/jemalloc/internal/arena.h:571:
> >> Failed assertion: "pageind >= map_bias" pid 21 (ls), uid 0: exited
> >> on signal 6 Abort trap
> >>
> >> It's happening every time with the command "ls".
> >>
> >> I tried to use the previous version of jemalloc (i.e reverting
> >> all the patches up to "Update jemalloc to version 4.0.0" included)
> >> and everything is working.
> >>
> >> Note that I'm using Freebsd as a Xen ARM guest although the only
> >> difference is the version of jemalloc (4.0.0 vs 3.6.0).
> >>
> >> Does anyone using arm64 have seen a similar ASSERT?
> >>
> >> BTW, is there any way to rebuild only the libc rather than doing
> >> make buildworld everytime I modified the jemalloc code?
> >>
> >> Regards,
> >>
> > 
> > This is a bug in the runtime linkers handling of tls. The attached
> > patch allows me to get to multiuser mode without anything hitting
> > the above assert.
> 
> I ended up to a similar patch during the week-end (see below).
> Although I was looking to the amd64/i386 definition of
> calculate_tls_offset which is the same as ARM64. I didn't understand
> why it's working for this architecture but not for ours.
> 
> Is there any possible bug in the amd64/i386 runtime too?

amd64 and i386 use a different tls variant. They place the thread
pointer at end of the buffer and allocate data before it, where arm64
places it at the start and allocates space after this.

I suspect it was previously working by chance.

There is a bug with all Variant I tls architectures (arm, arm64, mips,
and powerpc) where the thread pointer may be trashed. I expect to fix
this soon, and have a more complete fix committed soon after this.

(For those interested in the layout of the tls data there is a paper on
it at [1], with useful figures on pages 5 and 6).

Andrew

[1] http://www.akkadia.org/drepper/tls.pdf