Re: zfs and git upload-pack

On Sun, 7 Aug 2022 09:54:41 -0700
David Christensen <dpchrist@holgerdanske.com> wrote:

> On 8/7/22 01:28, Philipp wrote:
> > Hi all
> > 
> > I host a quite uncommon git repository mostly out of binary files. I
> > have the problem every time this repo is cloned the host allocate
> > memory and going to swap. This leads to the host being unusable and
> > need to force rebooted.
> > 
> > The repo is stored on a zfs and nullmounted in a jail to run the git
> > service over ssh. The host is a FreeBSD 13.1 with 4GB RAM and 4GB
> > swap.
> > 
> > What I have noticed is that the biggest memory consumtion is from
> > mmap() a pack file. For the given repo this has the size of 6,7G. I
> > suspect this file is mapped in memory but not correctly
> > handled/unmaped (by the kernel) when not enough memory is available.
> > 
> > I have tested some options to solve/workaround this issue:
> > 
> > * limit the zfs ARC size in loader.conf
> > * zfs set primarycache none for the dataset
> > * limit datasize, memoryuse and vmemoryuse via login.conf
> > * limit git packedGitLimit
> > 
> > None of them have solved the issue.
> > 
> > On my last test I have noticed that the memory is not freed after
> > the git commands are stopped. To free the memory I had to unmount
> > the zfs dataset:
> > 
> > satanist@confusion:~ $ freecolor -om
> >               total       used       free     shared    buffers
> > cached Mem:          3923       3881         41          0
> > 0 0 Swap:         4096        456       3639
> > satanist@confusion:~ $ sudo zfs umount zroot/data/git/bigrepository
> > satanist@confusion:~ $ freecolor -om
> >               total       used       free     shared    buffers
> > cached Mem:          3923        278       3645          0
> > 0 0 Swap:         4096        456       3639
> > 
> > Any idea how to debug this or what else I could try solve this?
> > 
> > Philipp  
> 
> 
> Add memory; the more, the better.

Can you borrow me your credit card?

Jokes aside. this repo gets cloned a few times a month. Currently the
Host dies because one client try to clone this repo. This should be work
with 4GB memory.

Philipp