syncing large mmaped files

[Konstantin Belousov]

On Thu, Oct 18, 2012 at 10:08:22AM +1000, Tristan Verniquet wrote:
> 
> I want to work with large (1-10G) files in memory but eventually sync
> them back out to disk. The problem is that the sync process appears to
> lock the file in kernel for the duration of the sync, which can run
> into minutes. This prevents other processes from reading from the file
> (unless they already have it mapped) for this whole time. Is there
> any way to prevent this? I think I read in a post somewhere about
> openbsd implementing partial-writes when it hits a file with lots of
> dirty pages in order to prevent this. Is there anything available for
> FreeBSD or is there another way around it?
>
No, currently the vnode lock is held exclusive for the whole duration
of the msync(2) syscall or its analog from the syncer.

Making a change to periodically drop the vnode lock in
vm_object_page_clean() might be possible, but requires the benchmarking
to make sure that we do not pessimize the common case. Also, this opens
a possibility for the vnode reclamation meantime.

Anyway, note that you cannot 'work with large files in memory', even if
you have enough RAM and no pressure to hold all the file pages resident.
The syncer will do a writeback periodically regardless of the application
calling msync(2) or not, with the interval of approximately 30 seconds.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20121018/59b4f16d/attachment.sig>