[PATCH] fadvise(2) system call

[John Baldwin]

On Friday, November 04, 2011 12:03:20 pm Alfred Perlstein wrote:
> * John Baldwin <jhb at FreeBSD.org> [111103 19:38] wrote:
> > On 11/2/11 1:20 PM, Alfred Perlstein wrote:
> > >* John Baldwin<jhb at freebsd.org>  [111102 11:05] wrote:
> > >>On Tuesday, November 01, 2011 7:56:48 pm Bruce Cran wrote:
> > >>>On 01/11/2011 17:08, John Baldwin wrote:
> > >>>>I had found it via the web: http://linux.die.net/man/2/fadvise
> > >>>>However, after further searching it appears to be stale (if you follow
> > >>>>it's cross-reference to madvise(2), that page only has links to
> > >>>>posix_fadvise() and not fadvise()).
> > >>>
> > >>>There's
> > >>>http://www.speedware.com/HPe3000_resources/MPE_to_HP-UX_cross-
> > >>reference/system_administration_cross-reference/cmd.html?cmdid=MS_1800
> > >>>for HP-UX ("*fadvise()* was derived by HP from the IEEE POSIX
> > >>>1003.1-2001 Standard"), though it also has posix_fadvise.
> > >>
> > >>Hmm, that one actually has an extra argument.  I'll just go with
> > >>posix_fadvise() for now.  Interesting that HP lets you OR together
> > >>two policies (so you can say both "I will access this file sequentially
> > >>and with noreuse").
> > >
> > >Makes sense for gzip/tar.
> > 
> > Ehh, quite possibly not for something that generic.  I think you only
> > want to do this if you have very specific knowledge about your access
> > pattern, and I do not think they are generically applicable.
> 
> You often spend time untarring the same tarball over and over 
> in your workflow John?

No, but many people will do a 'tar tvf' of a file after they generate it.  
Would be rather silly to force that to go to disk every time, especially given 
tar's format where it does not have a header like zip so you have to read the 
whole darn thing for a 'tar tvf'.

I think it would be fine to add flags to applications like 'tar' to allow 
users to alter their behavior in specific use cases when it makes sense.  
However, I think there are more workloads for 'tar' than the ones you are 
thinking of and we should be hesitant to change applications to use non-
default settings.  I do think FADV_SEQUENTIAL would be easy to add hints for, 
it is more FADV_NOREUSE and FADV_DONTNEED that I think warrant more caution as 
you can end up greatly hampering performance if you are not careful.  I think 
for the general case we should let pagedaemon manage this instead.  This is 
similar to the discussions about NUMA and the various notes from people who 
have worked with it in the past that having the kernel try to employ 
heuristisc often ends up breaking many things vs letting non-default behavior 
being driven by hints from users.

-- 
John Baldwin