Re: zfs and git upload-pack

On 8/7/22 01:28, Philipp wrote:
> Hi all
> 
> I host a quite uncommon git repository mostly out of binary files. I
> have the problem every time this repo is cloned the host allocate
> memory and going to swap. This leads to the host being unusable and
> need to force rebooted.
> 
> The repo is stored on a zfs and nullmounted in a jail to run the git
> service over ssh. The host is a FreeBSD 13.1 with 4GB RAM and 4GB swap.
> 
> What I have noticed is that the biggest memory consumtion is from mmap()
> a pack file. For the given repo this has the size of 6,7G. I suspect
> this file is mapped in memory but not correctly handled/unmaped (by the
> kernel) when not enough memory is available.
> 
> I have tested some options to solve/workaround this issue:
> 
> * limit the zfs ARC size in loader.conf
> * zfs set primarycache none for the dataset
> * limit datasize, memoryuse and vmemoryuse via login.conf
> * limit git packedGitLimit
> 
> None of them have solved the issue.
> 
> On my last test I have noticed that the memory is not freed after the
> git commands are stopped. To free the memory I had to unmount the zfs
> dataset:
> 
> satanist@confusion:~ $ freecolor -om
>               total       used       free     shared    buffers
> cached Mem:          3923       3881         41          0          0
>         0 Swap:         4096        456       3639
> satanist@confusion:~ $ sudo zfs umount zroot/data/git/bigrepository
> satanist@confusion:~ $ freecolor -om
>               total       used       free     shared    buffers
> cached Mem:          3923        278       3645          0          0
>         0 Swap:         4096        456       3639
> 
> Any idea how to debug this or what else I could try solve this?
> 
> Philipp


Add memory; the more, the better.


David