File tree replication in FreeBSD

[Bill Moran]

"Kentucky Mandeloid Mo." <ky at df.ru> wrote:
> On Sunday 27 June 2004 15:20, Luke Kearney wrote:
> > On Sun, 27 Jun 2004 13:48:07 +0400
> >
> > "Kentucky Mandeloid Mo." <ky at df.ru> spake thus:
> > > I'm looking for file replication solution for FreeBSD.
> > >
> > > I have a task to replicate about 11G of files with 500k total files
> > > count. I say replication not mirroring i.e. I need to copy modified files
> > > to slaves almost imidiately after the change.
> > > Tried to find some ready solution but fails.
> > > There is FAM from SGI but it fails to work properly on FreeBSD (it makes
> > > silly stat(2) calls).
> > > kevent(2) also don't have anough API to watch >= 500k files.
> > > So I see the only way is to have kernel module that watch kernel calls
> > > for file operations.
> > >
> > > So is there some other points to build a file replication?
> >
> > Can I suggest you investigate rsync ? It will allow you to replicate
> > file trees pretty much on demand across a LAN or indeed the internet
> > should you choose to. Works over SSH so it's nice and secure and if you
> > need virtually instant replication you could conceivably run this from a
> > cron job every N minutes.
> >
> > /usr/ports/net/rsync
> 
> Yes. I use rsync already and its a great tool.
> But actually its too slow with numbers of fles > 100k.
> Its takes more then 5mins to found just few files to update in list of 500k 
> files.
> 
> So I decide rsync to be a transport manager for my own replication tool. 
> Right now I'm writing a kernel module that catches all file syscalls that can 
> modify a file. Its almost done exept rmdir & unlink calls - I don't know how 
> to get a cwd from a thread. Function kern___getcwd sometimes fails with "not 
> a directory" error. 
> BTW, whats a good place where can I ask about kernel programming?

freebsd-hackers at freebsd.org is the best place I know of.

-- 
Bill Moran
Potential Technologies
http://www.potentialtech.com