rrdtool performance tuning (fwd)

[Alfred Perlstein]

* Dmitry Morozovsky <marck at rinet.ru> [071030 02:30] wrote:
> On Mon, 29 Oct 2007, Alfred Perlstein wrote:
> 
> AP> * Dmitry Morozovsky <marck at rinet.ru> [071029 12:44] wrote:
> AP> > 
> AP> > [hmm, after thinking a bit I decided it would be more appropriate here, in 
> AP> > stable@]
> AP> > 
> AP> > Dear colleagues,
> AP> > 
> AP> > any hints to tune rrdtool with ~30k rrd files (approx 2k target devices)?
> AP> > 
> AP> > machine is mostly IO-bound, showing 100% disk load with 8 or sometimes even 3 
> AP> > mB/s, 300-400 tps (it's 2 SATA300 disks in gmirror)
> AP> 
> AP> More ram?
> 
> Already 2G, mostly inactive/free.
> 
> AP> 
> AP> Turn off atime?
> 
> For sure, and even tried to move rrd data to smaller UFS2 - same result.
> 
> AP> Hash the data files into multiple directories to avoid having 2k files
> AP> in one dir.
> 
> Hmm, I thought 2k is not so much, especially where UFSDIRHASH is in place...
> 
> AP> Not sure how rrd tool works internally, but it might make sense
> AP> to see if you can use some layering library to force it to cache
> AP> some open files per process or something.
> 
> It seems it using a lot of mmap...

My try hacking the code and adding MAP_NOSYNC to the call to mmap(2).

Try hacking the code to keep the file mapped instead of open/closing
it on each access.




-Alfred