Possible problems with mmap/munmap on FreeBSD ...

[David Schultz]

On Tue, Mar 29, 2005, Richard Sharpe wrote:
> Hi,
> 
> I am having some problems with the tdb package on FreeBSD 4.6.2 and 4.10.
> 
> One of the things the above package does is:
> 
>    mmap the tdb file to a region of memory
>    store stuff in the region (memmov etc).
>    when it needs to extend the size of the region {
>      munmap the region
>      write data at the end of the file
>      mmap the region again with a larger size
>    }
> 
> What I am seeing is that after the munmap the data written to the region
> is gone.
> 
> However, if I insert an msync before the munmap, everything is nicely
> coherent. This seems odd (in the sense that it works without the msync
> under Linux).
> 
> The region is mmapped with:
> 
>    mmap(NULL, tdb->map_size,
>         PROT_READ|(tdb->read_only? 0:PROT_WRITE),
>         MAP_SHARED|MAP_FILE, tdb->fd, 0);
> 
> What I notice is that all the calls to mmap return the same address.
> 
> A careful reading of the man pages for mmap and munmap does not suggest
> that I am doing anything wrong.
> 
> Is it possible that FreeBSD is deferring flushing the dirty data, and then
> forgets to do it when the same starting address is used etc?

It looks like all of the underlying pages are getting invalidated
in vm_object_page_remove().  This is clearly the right thing to do
for private mappings, but it seems wrong for shared mappings.
Perhaps Alan has some insight.