Regarding zfs send / receive

[Joar Jegleim]

sounds like a good idea, I might look into that, thnx.
Terje: zpool.cache is only 860 bytes, I don't think that should cause any
problems (?)


-- 
----------------------
Joar Jegleim
Homepage: http://cosmicb.no
Linkedin: http://no.linkedin.com/in/joarjegleim
fb: http://www.facebook.com/joar.jegleim
AKA: CosmicB @Freenode

----------------------

On 5 April 2013 02:52, Waitman Gobble <gobble.wa at gmail.com> wrote:

> Waitman Gobble
> San Jose California USA
>
> On Apr 4, 2013 2:07 PM, "Joar Jegleim" <joar.jegleim at gmail.com> wrote:
> >
> > Hi Terje !
> > sorry for late reply, I've been checking my mail, forgetting that all my
> > mailing list mail are sorted into their own folders skipping inbox :p
> >
> > the zfs sync setup is a huge advantage over rsync simply because
> > incremental rsync of the volume takes ~12 hours, while the zfs
> differential
> > snapshot's usually take less than a minute . Though it's only ~1TB of
> data,
> > it's  more than 2 million jpegs which rsync have to stat ...
> > I'm guessing my predecessor who chose this setup, over for instance HAST,
> > didn't feel confident enough regarding HAST in production ( I'm looking
> > into that for a future solution) .
> >
> > There's no legacy stuff on the receiving end, old pools are deleted for
> > every sync. I haven't got my script here but google pointed me too
> > https://github.com/hoopty/zfs-sync/blob/master/zfs-sync which look like
> a
> > script very similar to the one I'm using .
> > In fact, I'm gonna take a closer look at that script and see what differs
> > from my script (apart from it being much prettier :p )
> > I didn't know about zpool.cache, gonna check that tomorrow, thanks.
> >
> >
> >
> > --
> > ----------------------
> > Joar
> > Jegleim
> > Homepage: http://cosmicb.no
> > Linkedin: http://no.linkedin.com/in/joarjegleim
> > fb: http://www.facebook.com/joar.jegleim
> > AKA: CosmicB @Freenode
> >
> > ----------------------
> >
> > On 2 April 2013 14:40, Terje Elde <terje at elde.net> wrote:
> >
> > > On 2. apr. 2013, at 13.44, Joar Jegleim wrote:
> > > > So my question(s) to the list would be:
> > > > In my setup have I taken the use case for zfs send / receive too far
> > > > (?) as in, it's not meant for this kind of syncing and this often, so
> > > > there's actually nothing 'wrong'.
> > >
> > > I'm not sure if you've taken it too far, but I'm not entirely sure if
> > > you're getting any advantage over using rsync or similar for this kind
> of
> > > thing.
> > >
> > > First two things that spring to mind:
> > >
> > > Do you have any legacy stuff on the receiving machine?  Things like
> > > physically removed old zpools, that are still in zpool.cache, seems to
> slow
> > > down various operations, including creation of new stuffs (such as the
> > > snapshots you receive).
> > >
> > > Also, you don't mention if you're deleting old snapshots on the
> receiving
> > > end?  If you're doing an incremental run every 15 minutes, that's
> something
> > > like 3000 snapshots pr. month, pr. filesystem.
> > >
> > > Terje
> > >
> > >
> >
>
> hi,
> i have a similar situation. its better to only rsync new stuff in this
> case, because you should know when somebody ads something new.
>
> for example, a user uploads 200 new images, these are marked 'to sync' and
> are transferred to the other servers. letting rsync figure out what's new
> just isnt practical.
>
> an idea, works for me. hope it helps.
>
> Waitman Gobble
> San Jose California _______________________________________________
> > freebsd-questions at freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> > To unsubscribe, send any mail to "
> freebsd-questions-unsubscribe at freebsd.org"
>