Re: Git and buildworld running at the same time

in my systems i've had buildworld failures with recent git. i didn't even know it does bg maint

and one other thing was that if i let git loose

and don't do

[core]
        packedGitWindowSize = 1m
        packedGitLimit = 1m
        preloadIndex = false
[diff]
        renameLimit = 1

it'll tell kernel to eat up my ram and cause userland to die during some particular unknown git operation

this is also on zfs and i'm not alone here, while ufs doesn't do this (in other people's machines)

i don't know where to point finger (mmap? slow io?) or how to test it in vm. or with what command. i noticed that lot of changes via pull in ports caused this 100% a time

seems like this needs several factors for perfect storm. note that git is only thing that can do this. or perhaps dovecot too, i configured that thing to not do mmap and / or cache

to this day i don't have proper solution to this

seems like slow io has part here

i could simulate that with geom (gnop maybe?) into a vm but i can't figure out how to have the proper stress test set for this. i tried several (memory) stress methods but they never do this

unsure what this really is. is it a feature or bug or how to limit it

i find it totally unacceptable that zfs is able to take any commands from userland that cause kernel to take over 95% of ram in few seconds. it should start waiting far before

this is not arc too. something in the kernel is allocating all of my ram from git fs operations and that sucks

maybe this thing is missing from zfs test suite? or maybe noone ran that thing on less than ideal machines?

funnily i'm able to do anything else on this c2d & 4g machine. just needs time, nothing fails

i have no idea how to solve this mystery. zfs has many tuning options, are some of them related to writes too? i haven't used any of them

funnily those options don't seem to slow git down, if you could say it's fast before

so any ideas?

before people tried to test this by lowering ram in vm, but if they had those on fast ssd's, it would never manifest

unless i'm mistaken here, this is slow io issue?

but how could system accept more that it can do?

i don't know how zfs works but can't it like put io operations into pause if it looks like disks are not keeping up?

just like cpu or ram limits are respected and nothing really goes "over the edge"

i expect this to bite users in future. especially corner case hw. this could be purposefully low end hw too

i could come back with a test case but i don't really have place where i can run things repeatedly into failure