Difference between p_vmspace quota between stable/11 and stable/10

[Konstantin Belousov]

On Wed, May 24, 2017 at 01:00:51AM -0700, Shrikanth Kamath wrote:
> I have a certain application(32 bit user space running in compat mode
> in 64 bit system X86) which does a fork, and the child process does a
> series of mmap calls as part of a scaling test. I am currently
> debugging an issue with this application which hits a ENOMEM when it
> is run on a stable/11 based setup and fails subsequent mmap and/or any
> malloc calls v/s stable/10 where the issue is not seen.. I probed the
> vm_map_find function using DTrace when "execname" was my application
> in question, and got these readings
> 
> fbt:kernel:vm_map_find:entry
> /self->trace == 1/ /*enabled only during sys_mmap call of this application */
> {
> @bytes[args[4]] = sum(args[4]);
> printf("request length [%x]", args[4]);
> }
> 
> For stable_10 --> Total of 124 requests (length requested was
> 0x500000) with the test successful
> 124 * 0x500000 (5MB) ~ 620MB
> 
> For stable_11 --> Total of 109 mmap requests
> (0x500000/0x200000/0x3ff000 are the different  vm_size_t length
> arguments in vm_map_find). The test fails after 386MB has been
> approved.
> 24 * 0x500000 (5MB) ~ 120MB
> 82 * 0x200000 (2MB) ~ 164MB
> 3 * 0x3ff000 (4MB) ~ 12MB
> 
> 
> The process parent rlimits are
> 
> # cat /proc/5058/rlimit
> 
> cpu -1 -1
> fsize -1 -1
> data 3221225472 3221225472
> stack 16777216 16777216
> core -1 -1
> rss 67108864 33265819648
> memlock 67108864 33265819648
> nproc 512 512
> nofile 1024 1024
> sbsize -1 -1
> vmem -1 -1
> npts -1 -1
> swap -1 -1
> kqueues -1 -1
> umtx -1 -1
> 
> The requests started failing in stable/11 with just 386 MB approved
> v/s stable/10 which was successful in approving ~620MB.
> 
> My stable/11 is from early days and is at GRN 302410 (probably 10 months old)
> Any pointers or tips on what to probe further will be very helpful. Is
> there any limits breach that I should probe further? The limits set
> when a process is forked?
> Should I probe the p->vmspace initiazliation?

I doubt that limits are relevant for your issue.  Look at the process address
map at the moment when the request failed, I suspect that it is fragmented.
Use procstat -v <pid> to examine the address space.  You may spawn the
tool from your program when mmap(2) fails.