Script to merge mailinglist archives

Joshua Tinnin krinklyfig at spymac.com
Thu Jan 27 05:26:31 PST 2005


On Thursday 27 January 2005 04:32 am, Giorgos Keramidas 
<keramida at ceid.upatras.gr> wrote:
> On 2005-01-27 13:06, Mikko Heiskanen <mikko at whitecortex.net> wrote:
> > I've found archives of freebsd-questions mailinglist very useful.
> > But I have found them difficult to search, as in one month there
> > can be many archives, not just one per month.
>
> If you have the mail archives of every week in Unix mbox format, you
> can simply `join' the archives by cat(1):
>
> 	% cd ~/mail-archive/freebsd/questions/2004/10
> 	% cat * > /tmp/freebsd-questions
>
> If that is not sufficient for your purpose or you just feel like a
> bit of extra fun is ok, you can use formail(1) and procmail(1) to do
> custom filtering of your own.  Just set up a custom filter list in a
> file of your choise, and pipe the messages to formail/procmail:
>
> 	% cd ~/mail-archive/freebsd/questions/2004/10
> 	% cat * | formail -s procmail /tmp/customfilter

Oh, OK, the OP is working with archives from the list site. if you have 
the archives in txt files from the 
http://lists.freebsd.org/pipermail/freebsd-questions/ page, you can do 
this, just for an example this is what I have in ~/tmp after unzipping 
the files:

% ls -l
total 22336
-rw-r--r--  1 krinklyfig  1001  6315547 Jan 27 04:58 2004-December.txt
-rw-r--r--  1 krinklyfig  1001  6173698 Jan 27 04:58 2004-November.txt
-rw-r--r--  1 krinklyfig  1001  5654084 Jan 27 04:58 2004-October.txt
-rw-r--r--  1 krinklyfig  1001  4622812 Jan 27 04:58 2004-September.txt

% cd ~/tmp
% find . -type f | grep "Oct\|Nov\|Dec" | xargs cat > 
fbsd-q-04-oct-dec-arc

% ls -l
total 40080
-rw-r--r--  1 krinklyfig  1001   6315547 Jan 27 04:58 2004-December.txt
-rw-r--r--  1 krinklyfig  1001   6173698 Jan 27 04:58 2004-November.txt
-rw-r--r--  1 krinklyfig  1001   5654084 Jan 27 04:58 2004-October.txt
-rw-r--r--  1 krinklyfig  1001   4622812 Jan 27 04:58 2004-September.txt
-rw-r--r--  1 krinklyfig  1001  18143329 Jan 27 05:21 
fbsd-q-04-oct-dec-arc

(this will wrap in this email, but you get the idea)

You can further customize what is concatenated by using different 
expressions in grep. If you want to search within the files, piping to 
procmail would probably work best, as already suggested, which could be 
added to the above command.

- jt


More information about the freebsd-questions mailing list