CTM index on servers

[Isaac (.ike) Levy]

On Jan 28, 2013, at 3:51 PM, Stephen Montgomery-Smith wrote:
> On 01/28/2013 01:29 PM, Isaac (.ike) Levy wrote:
>> With some sort of index, the ease-of-use functionality of c[v]sup could be trivially replicated in sh(1).
>> Even just as a prototype, this is a task I will enthusiastically do, even if only for my own use- (I started something already which culls FTP tarballs, a less than ideal solution).
>> 
>> What do yall' think?
> 
> The index idea would be very easy for me to implement.  Unless anyone
> has a serious objection, I will probably go ahead and implement it as
> above.  Except I will not include the time stamp in the file, as the
> file already has a modification time on it.  

Fair enough, it makes the index far simpler to parse.

> I will probably sign the
> index file as well, but I don't think it is necessary, as if that
> information is corrupted it is unlikely to do any damage.

This is correct, a .sig not necessary.

However if my program can check the sig, I can have my fetcher bail early if the sig isn't kosher- and therefore avoid possible bandwidth hogging/DOS- (my bandwidth, or the project servers!).

> But ... I use a script like this:
> 
> I store all the ctm files in /usr/home/stephen/ctm or subdirectories
> thereof.  This script sees what is already there, and then tries to
> fetch any future ctm deltas.
> 
> 
> #!/bin/sh
> 
> cd /usr/home/stephen/ctm/incoming
> 
> bases=`find /usr/home/stephen/ctm -type f |
>       sed -n -E -e 's+.*/++' -e 's/\.[[:digit:]]{4}\.gz//p' | sort -u`
> 
> for b in $bases; do
>  while true; do
>    index=`find /usr/home/stephen/ctm -type f -name "$b*" |
>           sed -n -E 's/.*([[:digit:]]{4}).*/\1/p' |
>           sort -n | tail -1 | xargs expr 1 + | xargs printf "%04d"`
>    fetch ftp://ftp.freebsd.org/pub/CTM/$b/$b.$index.gz || break
>  done
> done
> 
> echo
> ls

Thanks for that shell bit!!!

Best,
.ike