Why use jemalloc bundled with Mozilla?

[Jan Beich]

"Mikhail T." <mi+thun at aldan.algebra.com> writes:

> The system malloc on FreeBSD-10.x is jemalloc-3.4.x. Why do Mozilla ports build
> with "--enable-jemalloc" turned on despite that?

Firefox devs work with jemalloc upstream, so newer version is preferred.
Some changes in 37.0 make it harder to support in-base version:

# stats.bookkeeping, available as stats.metadata since jemalloc-4.0.0
https://bugzilla.mozilla.org/show_bug.cgi?id=899126
https://github.com/jemalloc/jemalloc/issues/163

# junk:free, available since jemalloc-4.0.0
https://bugzilla.mozilla.org/show_bug.cgi?id=1108045
https://github.com/jemalloc/jemalloc/pull/172

# *allocx() is N/A before jemalloc-3.4.x while *allocm() is N/A in 4.0.0
https://hg.mozilla.org/mozilla-central/diff/b5b1a8e8aee4/memory/build/mozjemalloc_compat.c
https://hg.mozilla.org/mozilla-central/diff/96a98f1d5aa8/configure.in
https://github.com/jemalloc/jemalloc/commit/9790b9667fd975b1f9a4f108f9d0a20ab265c6b6

> Is it just a typo in bsd.gecko.mk and the fix as simple as:

As nallocx() is not available on FreeBSD 10.x non-standard API usage
would be silently disabled. As stats.bookkeeping isn't available in any
released jemalloc version while CTL_GET macro lacks error-checking it'd
return garbage in about:memory on FreeBSD 11.0C.

However, using bundled jemalloc leads to unnecessary separation for
memory accounting e.g., try looking at stderr after

  $ MALLOC_CONF=stats_print:true firefox ...
  $ JE_MALLOC_CONF=stats_print:true firefox ...

>
>     --- /usr/ports/Mk/bsd.gecko.mk  (revision 390575)
>     +++ /usr/ports/Mk/bsd.gecko.mk  (working copy)
>     @@ -130,7 +130,7 @@
>      
>      # use jemalloc 3.0.0 API for stats/tuning
>      MOZ_EXPORT+=   MOZ_JEMALLOC3=1
>     -.if ${OSVERSION} < 1000012 || ${MOZILLA_VER:R:R} >= 37
>     +.if ${OSVERSION} < 1000012 && ${MOZILLA_VER:R:R} >= 37
>      MOZ_OPTIONS+=  --enable-jemalloc
>      .endif
>      
>
> or is there more to it? Thanks! Yours,

If you're eager for preview try (before jasone@ tags and imports 4.0.0)

  # See VERSION or FREEBSD-upgrade under contrib/jemalloc/ to reproduce
  $ fetch https://people.freebsd.org/~jbeich/jemalloc_update.20150716.diff.xz
  $ xzcat jemalloc_update.20150716.diff.xz | (cd /usr/src; patch -Efsp0)

then apply the following ports' patch, rebuild firefox and check "Measure"
in about:memory makes sense. heap-* reporters should be closer to what
MALLOC_CONF=stats_print:true shows.

Index: Mk/bsd.gecko.mk
===================================================================
--- Mk/bsd.gecko.mk	(revision 392295)
+++ Mk/bsd.gecko.mk	(working copy)
@@ -130,9 +130,6 @@ LDFLAGS+=		-L${LOCALBASE}/lib -Wl,-rpath,${PREFIX}
 
 # use jemalloc 3.0.0 API for stats/tuning
 MOZ_EXPORT+=	MOZ_JEMALLOC3=1
-.if ${OSVERSION} < 1000012 || ${MOZILLA_VER:R:R} >= 37
-MOZ_OPTIONS+=	--enable-jemalloc
-.endif
 
 # Standard depends
 _ALL_DEPENDS=	cairo event ffi graphite harfbuzz hunspell icu jpeg nspr nss opus png pixman soundtouch sqlite vorbis vpx
Index: www/firefox/files/patch-jemalloc4
===================================================================
--- www/firefox/files/patch-jemalloc4	(revision 0)
+++ www/firefox/files/patch-jemalloc4	(working copy)
@@ -0,0 +1,11 @@
+--- memory/build/mozjemalloc_compat.c~
++++ memory/build/mozjemalloc_compat.c
+@@ -150,7 +150,7 @@ jemalloc_stats_impl(jemalloc_stats_t *st
+   CTL_GET("stats.allocated", allocated);
+   CTL_GET("stats.mapped", mapped);
+   CTL_GET("opt.lg_chunk", lg_chunk);
+-  CTL_GET("stats.bookkeeping", stats->bookkeeping);
++  CTL_GET("stats.metadata", stats->bookkeeping);
+ 
+   /* get the summation for all arenas, i == narenas */
+   CTL_I_GET("stats.arenas.0.pdirty", pdirty, narenas);
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 602 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-gecko/attachments/20150716/0009dbb9/attachment.bin>