Giving up on x buffers - losing files

[Don Lewis]

On 26 Jun, Arjan van Leeuwen wrote:
> I've had this problem for a long time with 5.x, but it doesn't seem to  
> happen that often. Today it bit me hard.
> 
> Sometimes, particularly after doing a lot of file writes (i.e. compiling a  
> lot of ports, building world and mergemastering, etc), I get the 'Giving  
> up on x buffers' message on shutdown, and my filesystems come up dirty  
> when I restart.
> 
> This wouldn't be such an enormous problem, if it wouldn't always erase the  
> files I changed most recently. The files are simply reduced to 0 bytes. My  
> configuration files for Opera and KDE have been victim to this more than  
> once (because Opera writes to the file on exit, for example), but today,  
> it was /etc/master.passwd that was reduced to 0 bytes (because I had just  
> changed something in it).
> 
> I understand that turning of write caching might improve the situation,  
> but it also makes my system a lot slower, and I don't like that on my  
> desktop system.
> 
> So, why does this happen? And how do I prevent it from happening? This  
> definitely does _not_ sound like something I want my servers to do when  
> 5.x goes -STABLE.

I've mentioned this a couple of times on this list in the last six
months or so.  The last time was in the last couple of weeks.  I can
reliably trigger this problem with mergemaster.

I'm pretty sure that the problem relates to soft updates and how the
file system syncer is shut down, which leaves unresolved dependencies
that keep a number of dirty blocks from being flushed to disk at the end
of the system shutdown.

I have some ideas on how to fix the problem, but I haven't had the time
to work on it and nobody else has stepped up with a fix.

I am able to reliably work around the problem by running the sync
command and waiting a short while after running mergemaster and before
shutting down or rebooting the machine.