svn commit: r280327 - in head/sys: kern vm

Mon Mar 30 17:05:50 UTC 2015

On 03/30/2015 01:25, Don Lewis wrote:
> On 28 Mar, Alan Cox wrote:
>> On 03/28/2015 12:29, Bryan Drewery wrote:
>>> On 3/27/2015 9:41 PM, Don Lewis wrote:
>>>> On 21 Mar, Alan Cox wrote:
>>>>> Author: alc
>>>>> Date: Sat Mar 21 17:56:55 2015
>>>>> New Revision: 280327
>>>>> URL: https://svnweb.freebsd.org/changeset/base/280327
>>>>>
>>>>> Log:
>>>>>   Introduce vm_object_color() and use it in mmap(2) to set the color of
>>>>>   named objects to zero before the virtual address is selected.  Previously,
>>>>>   the color setting was delayed until after the virtual address was
>>>>>   selected.  In rtld, this delay effectively prevented the mapping of a
>>>>>   shared library's code section using superpages.  Now, for example, we see
>>>>>   the first 1 MB of libc's code on armv6 mapped by a superpage after we've
>>>>>   gotten through the initial cold misses that bring the first 1 MB of code
>>>>>   into memory.  (With the page clustering that we perform on read faults,
>>>>>   this happens quickly.)
>>>>>   
>>>>>   Differential Revision:	https://reviews.freebsd.org/D2013
>>>>>   Reviewed by:	jhb, kib
>>>>>   Tested by:	Svatopluk Kraus (armv6)
>>>>>   MFC after:	6 weeks
>>>>>
>>>>> Modified:
>>>>>   head/sys/kern/kern_exec.c
>>>>>   head/sys/vm/vm_fault.c
>>>>>   head/sys/vm/vm_mmap.c
>>>>>   head/sys/vm/vm_object.h
>>>>>   head/sys/vm/vnode_pager.c
>>>> This change appears to have partially broken package building.
>>>>
>>>> I recently set up a package building machine running 11.0-CURRENT
>>>> r280642 and found that it was unable to build openjdk7 packages inside
>>>> FreeBSD 8.4 and 9.3 poudriere jails, for both i386 and amd64.
>>>>
>>>> [snip]
>>>> Compiling ../generated/adfiles/ad_x86_64_gen.cpp
>>>> rm -f ad_x86_64_gen.o
>>>> c++ -D_ALLBSD_SOURCE -D_GNU_SOURCE -DAMD64 -DPRODUCT -I. -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/precompiled -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/cpu/x86/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os_cpu/bsd_x86/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os/bsd/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os/posix/vm -I../generated -DHOTSPOT_RELEASE_VERSION="\"24.76-b04\"" -DHOTSPOT_BUILD_TARGET="\"product\"" -DHOTSPOT_BUILD_USER="\"root\"" -DHOTSPOT_LIB_ARCH=\"amd64\" -DHOTSPOT_VM_DISTRO="\"OpenJDK\"" -O2 -pipe -fstack-protector -fno-strict-aliasing -DTARGET_OS_FAMILY_bsd -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_bsd_x86 -DTARGET_OS_ARCH_MODEL_bsd_x86_64 -DTARGET_COMPILER_gcc -DCOMPILER2 -DCOMPILER1  !
>  -fno!
>>>>  -rtti -fno-exceptions -pthread -fcheck-new -m64 -pipe -DTARGET_OS_FAMILY_bsd -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_bsd_x86 -DTARGET_OS_ARCH_MODEL_bsd_x86_64 -DTARGET_COMPILER_gcc -DCOMPILER2 -DCOMPILER1 -fPIC -fno-rtti -fno-exceptions -pthread -fcheck-new -m64 -pipe -O3 -fno-strict-aliasing -DVM_LITTLE_ENDIAN -D_LP64=1 -fno-omit-frame-pointer -DINCLUDE_TRACE=1  -Wpointer-arith -Wconversion -Wsign-compare    -c  -fpch-deps -MMD -MP -MF ../generated/dependencies/ad_x86_64_format.o.d -o ad_x86_64_format.o ../generated/adfiles/ad_x86_64_format.cpp 
>>>> c++ -D_ALLBSD_SOURCE -D_GNU_SOURCE -DAMD64 -DPRODUCT -I. -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/precompiled -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/cpu/x86/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os_cpu/bsd_x86/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os/bsd/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os/posix/vm -I../generated -DHOTSPOT_RELEASE_VERSION="\"24.76-b04\"" -DHOTSPOT_BUILD_TARGET="\"product\"" -DHOTSPOT_BUILD_USER="\"root\"" -DHOTSPOT_LIB_ARCH=\"amd64\" -DHOTSPOT_VM_DISTRO="\"OpenJDK\"" -O2 -pipe -fstack-protector -fno-strict-aliasing -DTARGET_OS_FAMILY_bsd -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_bsd_x86 -DTARGET_OS_ARCH_MODEL_bsd_x86_64 -DTARGET_COMPILER_gcc -DCOMPILER2 -DCOMPILER1  !
>  -fno!
>>>>  -rtti -fno-exceptions -pthread -fcheck-new -m64 -pipe -DTARGET_OS_FAMILY_bsd -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_bsd_x86 -DTARGET_OS_ARCH_MODEL_bsd_x86_64 -DTARGET_COMPILER_gcc -DCOMPILER2 -DCOMPILER1 -fPIC -fno-rtti -fno-exceptions -pthread -fcheck-new -m64 -pipe -O3 -fno-strict-aliasing -DVM_LITTLE_ENDIAN -D_LP64=1 -fno-omit-frame-pointer -DINCLUDE_TRACE=1  -Wpointer-arith -Wconversion -Wsign-compare    -c  -fpch-deps -MMD -MP -MF ../generated/dependencies/ad_x86_64_gen.o.d -o ad_x86_64_gen.o ../generated/adfiles/ad_x86_64_gen.cpp 
>>>> Compiling ../generated/adfiles/ad_x86_64_misc.cpp
>>>> rm -f ad_x86_64_misc.o
>>>> c++ -D_ALLBSD_SOURCE -D_GNU_SOURCE -DAMD64 -DPRODUCT -I. -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/precompiled -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/cpu/x86/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os_cpu/bsd_x86/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os/bsd/vm -I/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/os/posix/vm -I../generated -DHOTSPOT_RELEASE_VERSION="\"24.76-b04\"" -DHOTSPOT_BUILD_TARGET="\"product\"" -DHOTSPOT_BUILD_USER="\"root\"" -DHOTSPOT_LIB_ARCH=\"amd64\" -DHOTSPOT_VM_DISTRO="\"OpenJDK\"" -O2 -pipe -fstack-protector -fno-strict-aliasing -DTARGET_OS_FAMILY_bsd -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_bsd_x86 -DTARGET_OS_ARCH_MODEL_bsd_x86_64 -DTARGET_COMPILER_gcc -DCOMPILER2 -DCOMPILER1  !
>  -fno!
>>>>  -rtti -fno-exceptions -pthread -fcheck-new -m64 -pipe -DTARGET_OS_FAMILY_bsd -DTARGET_ARCH_x86 -DTARGET_ARCH_MODEL_x86_64 -DTARGET_OS_ARCH_bsd_x86 -DTARGET_OS_ARCH_MODEL_bsd_x86_64 -DTARGET_COMPILER_gcc -DCOMPILER2 -DCOMPILER1 -fPIC -fno-rtti -fno-exceptions -pthread -fcheck-new -m64 -pipe -O3 -fno-strict-aliasing -DVM_LITTLE_ENDIAN -D_LP64=1 -fno-omit-frame-pointer -DINCLUDE_TRACE=1  -Wpointer-arith -Wconversion -Wsign-compare    -c  -fpch-deps -MMD -MP -MF ../generated/dependencies/ad_x86_64_misc.o.d -o ad_x86_64_misc.o ../generated/adfiles/ad_x86_64_misc.cpp 
>>>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/compiler/abstractCompiler.cpp:1: fatal error: had to relocate PCH
>>>> compilation terminated.
>>>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/utilities/accessFlags.cpp:1: fatal error: had to relocate PCH
>>>> compilation terminated.
>>>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'abstractCompiler.o' failed
>>>> gmake[6]: *** [abstractCompiler.o] Error 1
>>>> gmake[6]: *** Waiting for unfinished jobs....
>>>> ../generated/adfiles/ad_x86_64.cpp:1: fatal error: had to relocate PCH
>>>> compilation terminated.
>>> Are you using ccache?
>>>
>>>> ../generated/adfiles/ad_x86_64_clone.cpp:1: fatal error: had to relocate PCH
>>>> compilation terminated.
>>>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'accessFlags.o' failed
>>>> gmake[6]: *** [accessFlags.o] Error 1
>>>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'ad_x86_64_clone.o' failed
>>>> gmake[6]: *** [ad_x86_64_clone.o] Error 1
>>>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'ad_x86_64.o' failed
>>>> gmake[6]: *** [ad_x86_64.o] Error 1
>>>> ../generated/adfiles/ad_x86_64_expand.cpp:1: fatal error: had to relocate PCH
>>>> compilation terminated.
>>>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'ad_x86_64_expand.o' failed
>>>> gmake[6]: *** [ad_x86_64_expand.o] Error 1
>>>> ../generated/adfiles/ad_x86_64_gen.cpp:1: fatal error: had to relocate PCH
>>>> compilation terminated.
>>>> ../generated/adfiles/ad_x86_64_misc.cpp:1: fatal error: had to relocate PCH
>>>> compilation terminated.
>>>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'ad_x86_64_gen.o' failed
>>>> gmake[6]: *** [ad_x86_64_gen.o] Error 1
>>>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'ad_x86_64_misc.o' failed
>>>> gmake[6]: *** [ad_x86_64_misc.o] Error 1
>>>> ../generated/adfiles/ad_x86_64_format.cpp:1: fatal error: had to relocate PCH
>>>> compilation terminated.
>>>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/rules.make:149: recipe for target 'ad_x86_64_format.o' failed
>>>> gmake[6]: *** [ad_x86_64_format.o] Error 1
>>>> gmake[6]: Leaving directory '/wrkdirs/usr/ports/java/openjdk7/work/openjdk/build/bsd-amd64/hotspot/outputdir/bsd_amd64_compiler2/product'
>>>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/makefiles/top.make:128: recipe for target 'the_vm' failed
>>>> gmake[5]: *** [the_vm] Error 2
>>>> gmake[5]: Leaving directory '/wrkdirs/usr/ports/java/openjdk7/work/openjdk/build/bsd-amd64/hotspot/outputdir/bsd_amd64_compiler2/product'
>>>> /wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make/bsd/Makefile:292: recipe for target 'product' failed
>>>> gmake[4]: *** [product] Error 2
>>>> gmake[4]: Leaving directory '/wrkdirs/usr/ports/java/openjdk7/work/openjdk/build/bsd-amd64/hotspot/outputdir'
>>>> Makefile:191: recipe for target 'generic_build2' failed
>>>> gmake[3]: *** [generic_build2] Error 2
>>>> gmake[3]: Leaving directory '/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make'
>>>> Makefile:151: recipe for target 'product' failed
>>>> gmake[2]: *** [product] Error 2
>>>> gmake[2]: Leaving directory '/wrkdirs/usr/ports/java/openjdk7/work/openjdk/hotspot/make'
>>>> make/hotspot-rules.gmk:111: recipe for target 'hotspot-build' failed
>>>> gmake[1]: *** [hotspot-build] Error 2
>>>> gmake[1]: Leaving directory '/wrkdirs/usr/ports/java/openjdk7/work/openjdk'
>>>> Makefile:251: recipe for target 'build_product_image' failed
>>>> gmake: *** [build_product_image] Error 2
>>>> ===> Compilation failed unexpectedly.
>>>>
>>>> I was not seeing this problem on my older package builder running
>>>> 10.1-STABLE.  Since this problem has not shown up on the FreeBSD package
>>>> building cluster, I got suspicious that the change was quite recent.
>>>>
>>>> This old gcc bug report:
>>>> <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=14940> led me to suspect
>>>> mmap().
>>>>
>>>> The old gcc source file /usr/src/contrib/gcc/ggc-common.c does a couple
>>>> of mmap() calls.   Tne first is in mmap_gt_pch_get_address() where a
>>>> NULL first argument is used.  The address that gets returned is stashed
>>>> away and the region is unmapped.  Then a later call in
>>>> mmap_gt_pch_use_address() passes this saved  address to mmap() as a
>>>> hint.  It expects the mapped region to get mapped to the same base
>>>> address.  If this does not happen, the above error is the result.
>>> I don't know what I'm talking about but that doesn't sound like a very
>>> safe assumption for the gcc code to make.
>>
>> Your intuition is correct.  It is not.  And, in fact, under Solaris and
>> Linux, gcc does not make this assumption.
>>
>> I suspect that the solution used by gcc under Solaris would work for
>> us.  However, that would entail modifying gcc in older branches.
> After digging in to this today, I'm not sure that the Solaris solution
> would work.

Actually, based on what you've written below, the solution in
config/host-solaris.c should work.  That solution uses mincore(2) to
verify that the desired addresses are not in use and reissues the
mmap(2) call with MAP_FIXED if they are not.  MAP_FIXED overrides the
automatic realignment that moves the mapping up to 0x8115F4000.  This
should work on any version of FreeBSD.

In FreeBSD 10+, we have a new flag MAP_EXCL that eliminates the need for
the mincore(2) calls and the second call to mmap(2).  The first call can
safely specify MAP_EXCL | MAP_FIXED.

> On amd64, with an amd64 jail, and kernel rev r280326, I observe the
> following:
> 	One gcc process calls mmap() with addr=0 and len=0x657a000, and
>         the value 0x811400000 is returned.  Subsequent gcc processes
>         call mmap() with addr=0x811400000 and len=0x657a000, getting
>         0x811400000 in return.
>
> With kernel rev r280327 I see:
> 	One gcc process calls mmap() with addr=0 and len=0x657a000, and
> 	the value 0x811400000 is returned.  Subsequent gcc processes
> 	call mmap() with addr=0x811400000 and len=0x657a000, getting
> 	0x8115f4000 in return.  What I later noticed is that the subsequent
> 	calls are passing offset=0x1f4000.  Not so coincidentally,
> 	0x811400000 + 0x1f4000 = 0x8115F4000.
>
> My first attempt at a fix subtracted offset from address, but the mmap()
> return value changed to 0x8113f4000 instead of the 0x811400000 I was
> expecting.  It looked to me like the code must be doing superpage
> alignment on the start of the file and then adding the offset to get the
> start of the mapped region.
>
> This somewhat hacking patch disables this alignment if a non-zero
> address is passed as a hint, and allows the code to make the start of
> the mapped region match the hint.  With this patch, I've been able to
> build openjdk7 in a FreeBSD 9.3 amd64 jail.
>
> Index: vm_mmap.c
> ===================================================================
> --- vm_mmap.c	(revision 280327)
> +++ vm_mmap.c	(working copy)
> @@ -325,6 +325,8 @@
>  		 * There should really be a pmap call to determine a reasonable
>  		 * location.
>  		 */
> +		if (addr != 0 && (flags & MAP_ANON) == 0)
> +			flags |= MAP_RESERVED0100;
>  		PROC_LOCK(td->td_proc);
>  		if (addr == 0 ||
>  		    (addr >= round_page((vm_offset_t)vms->vm_taddr) &&
> @@ -1665,6 +1667,8 @@
>  		else if ((flags & MAP_ALIGNMENT_MASK) != 0)
>  			findspace = VMFS_ALIGNED_SPACE(flags >>
>  			    MAP_ALIGNMENT_SHIFT);
> +		else if ((flags & MAP_RESERVED0100) != 0)
> +			findspace = VMFS_ANY_SPACE;
>  		else
>  			findspace = VMFS_OPTIMAL_SPACE;
>  		rv = vm_map_find(map, object, foff, addr, size,
>
>         
>