Running rsnapshot via cron reboots the machine

[David N]

2008/12/18 Mel <fbsd.questions at rachie.is-a-geek.net>:
> On Wednesday 17 December 2008 11:40:00 David N wrote:
>> 2008/12/17 Mel <fbsd.questions at rachie.is-a-geek.net>:
>> > On Monday 15 December 2008 18:56:46 David N wrote:
>> >> Hi,
>> >>
>> >> I have a machine
>> >> AMD Sepron LE-1150
>> >> ASUS M2A-VM
>> >> 1GB RAM ECC
>> >> 2x SATA 300GB
>> >>
>> >> in a RAID 1 (gmirror).
>> >> 7.0-RELEASE-p2 AMD64 generic kernel
>> >>
>> >> it was doing backups via bacula to an external disk
>> >> USB 2.0 SATA disk, and it was working well. (GLabel) /dev/ufs/BackupDisk
>> >>
>> >> I changed to rsnapshot recently, with the External HDD in glabel +
>> >> gjournal (/dev/da0s1.journal -> /dev/ufs/BackupDisk) and it will
>> >> reboot the machine roughly 30 minutes after the rsnapshot starts via
>> >> CRON.
>> >
>> > Able to get any crash dumps? [1] I doubt it calls reboot system call
>> > after 30 minutes and if it's a heating issue, then it would power down
>> > not reboot. So, kernel is probably panicing.
>> >
>> > [1]
>> > http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kern
>> >eldebug.html
>> >
>> > --
>> > Mel
>> >
>> > Problem with today's modular software: they start with the modules
>> >    and never get to the software part.
>>
>> I found something in the vmcore.0
>>
>> panic: Journal overflow (joffset=499758276096 active=498475869184
>> inactive=499755984896)
>> cpuid = 0
>> Uptime: 16h7m11s
>>
>> I tried kgdb on on the vmcore but it didn't work, I had -p2 installed,
>> but compiled p6 so it might of overwrittin things in /usr/obj
>>
>> [GDB will not be able to debug user-mode threads:
>> /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
>> GNU gdb 6.1.1 [FreeBSD]
>> Copyright 2004 Free Software Foundation, Inc.
>> GDB is free software, covered by the GNU General Public License, and you
>> are welcome to change it and/or distribute copies of it under certain
>> conditions. Type "show copying" to see the conditions.
>> There is absolutely no warranty for GDB.  Type "show warranty" for details.
>> This GDB was configured as "amd64-marcel-freebsd".
>> Cannot access memory at address 0x0
>>
>>
>> The journal was set to 2GB on the 400GB USB attached disk. (/dev/da0)
>>
>> I just formatted the disk without gjournal and see how that goes. I
>> guess i can't use gjournal over USB? I have gjournal running on
>> another server (gmirror + gjournal) and i thrash it pretty hard
>> without any problems.
>
> It should not panic, but a journal overflow is more likely with USB, cause of
> the lower write speed (the journal fills faster then it's being emptied).
> Your best bet is to reproduce the panic using the sources that match the
> kernel and file a PR and/or post to freebsd-fs list to find out if there are
> people with similar problems/usage cases. It could be a tunable that you
> missed or that it's a known issue.
>
> --
> Mel
>
> Problem with today's modular software: they start with the modules
>    and never get to the software part.
>

There are people with similar problems already reported.
http://www.freebsd.org/cgi/query-pr.cgi?pr=127420

I tried the tunables
kern.geom.journal.force_switch=50
kern.geom.journal.cache.switch=75
which made it crash even faster, in a few minutes and even corrupted
the journal. I would test it out more, but its a production server
which needs to be up and running.

At the moment its just UFS+glabel, I'll try again when 7.1 comes out.

Thank you for your help.

Regards
David N