Synchronizing directory hierarchies via SFTP

[Michael Ross]

Am .12.2015, 01:13 Uhr, schrieb Patrick Hess <patrickhess at gmx.net>:

> Michael Ross wrote:
>> Am .12.2015, 00:43 Uhr, schrieb Patrick Hess <patrickhess at gmx.net>:
>> > Just came across LFTP, http://lftp.yar.ru. According to the feature  
>> list,
>> > it has support for SFTP and also comes with a "built-in mirror and
>> > reverse mirror" option.
>>
>> lftp is used a lot here to mirror directories via sftp. Would recommend.
>
> Tried it today, works like a charm. In comparison to csync, I noticed
> that the initial directory traversal in particular is a lot faster
> with LFTP. And since there's still a big question mark behind sshfs
> regarding its stability on FreeBSD, I'll just settle for LFTP and
> call it a day.
>
>> Check out the fine-tuning available with "set -a".
>
> Wow, that's a lot of stuff to mess up... Maybe some other day.
> So far, the defaults seem to work just fine for me.
>
> Patrick
>
> PS: Here's the quick'n'dirty Expect script I hacked together,
>     in case someone has a use for it.
> _______________________________________________________________________
>
> expect -c '
>    set timeout -1;
>
>    spawn lftp -u '"$SFTP_USERNAME"' sftp://'"$SFTP_SERVER"';
>
>    expect "Passwor\[dt\]:";
>    send "'"$SFTP_PASSWORD"'\r";
>
>    expect "lftp *> ";
>    send "mirror -v --reverse --dereference --delete --overwrite  
> --exclude-glob *.swp '"$LOCAL_DIR"' '"$REMOTE_DIR"'\r"
>
>    expect "lftp *> ";
>    send "quit\r";
>
>    expect eof;
>    '
> _______________________________________________________________________


You can put it in a file

	#!/usr/local/bin/lftp
	open sftp://user:pass@host
	mirror -v --reverse --dereference ...

and run it either per

	lftp -f your-file

or chmod +x it and go

	./your-file


As for settings, I'd take a look at least at the settings for net:,
you can grep for those

	set -a | grep net

Specifically:
	net:max-retries 1000
	net:reconnect-interval-base 30
	net:reconnect-interval-max 600
	net:reconnect-interval-multiplier 1.5
means, e. g. for an unreachable server,
lftp will try reconnecting every interval*multiplier seconds, until the  
delay has reached
interval-max seconds,
and *repeat this max-retries times*. That gives you about a week before  
the script actually exits.



Regards,

Michael