reproducible panic every day at 03:02, probably triggered by daily periodic scipts - help
Anton Shterenlikht
mexas at bris.ac.uk
Wed Mar 12 12:14:11 UTC 2014
>From be at 0x20.net Thu Mar 6 22:02:56 2014
>
>On Thu, Mar 06, 2014 at 12:59:14AM -0800, Anton Shterenlikht wrote:
>> In my initial PR (sparc64 r261798),
>>
>> http://www.freebsd.org/cgi/query-pr.cgi?pr=187080
>>
>> I said that rsync was triggering this panic.
>> While true, I now see that there's more to it.
>> I disabled the rsync, and the cron jobs.
>> Still I get exactly the same panic every
>> night at 03:02:
>>
>> # grep Dumptime /var/crash/*
>> /var/crash/info.0: Dumptime: Wed Feb 26 10:10:51 2014
>> /var/crash/info.1: Dumptime: Thu Feb 27 03:02:14 2014
>> /var/crash/info.2: Dumptime: Fri Feb 28 03:02:29 2014
>> /var/crash/info.3: Dumptime: Sat Mar 1 03:02:25 2014
>> /var/crash/info.4: Dumptime: Tue Mar 4 03:02:01 2014
>> /var/crash/info.5: Dumptime: Wed Mar 5 03:02:05 2014
>> /var/crash/info.6: Dumptime: Thu Mar 6 03:02:11 2014
>> /var/crash/info.last: Dumptime: Thu Mar 6 03:02:11 2014
>> #
>>
>> This is likely triggered by one of
>> the daily periodic scipts,
>> after about 1 min from start:
>>
>> # grep daily /etc/crontab
>> # Perform daily/weekly/monthly maintenance.
>> 1 3 * * * root periodic daily
>> #
>>
>> but which one?
>
>Some time ago I had a similar problem with 8.x. Setting
>
>vm.kmem_size="512M"
>vm.kmem_size_max="512M"
>
>in loader.conf helped. It's just a wild guess but might help.
This didn't make any difference.
However, I noticed that the panics were
happening more and more often. I started
suspecting a disk failure, so decided to
do a full integrity check with dd, got:
17054+0 records in
17054+0 records out
17882415104 bytes transferred in 1035.889679 secs (17262857 bytes/sec)
(ada1:ata2:0:1:0): READ_DMA. ACB: c8 00 80 93 9c 42 00 00 00 00 00 00
(ada1:ata2:0:1:0): CAM status: ATA Status Error
(ada1:ata2:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 40 (UNC )
(ada1:ata2:0:1:0): RES: 51 40 21 94 9c 02 02 00 00 00 00
(ada1:ata2:0:1:0): Retrying command
(ada1:ata2:0:1:0): READ_DMA. ACB: c8 00 80 93 9c 42 00 00 00 00 00 00
(ada1:ata2:0:1:0): CAM status: ATA Status Error
(ada1:ata2:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 40 (UNC )
(ada1:ata2:0:1:0): RES: 51 40 21 94 9c 02 02 00 00 00 00
(ada1:ata2:0:1:0): Retrying command
(ada1:ata2:0:1:0): READ_DMA. ACB: c8 00 80 93 9c 42 00 00 00 00 00 00
(ada1:ata2:0:1:0): CAM status: ATA Status Error
(ada1:ata2:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 40 (UNC )
(ada1:ata2:0:1:0): RES: 51 40 21 94 9c 02 02 00 00 00 00
(ada1:ata2:0:1:0): Retrying command
(ada1:ata2:0:1:0): READ_DMA. ACB: c8 00 80 93 9c 42 00 00 00 00 00 00
(ada1:ata2:0:1:0): CAM status: ATA Status Error
(ada1:ata2:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 40 (UNC )
(ada1:ata2:0:1:0): RES: 51 40 21 94 9c 02 02 00 00 00 00
(ada1:ata2:0:1:0): Retrying command
(ada1:ata2:0:1:0): READ_DMA. ACB: c8 00 80 93 9c 42 00 00 00 00 00 00
(ada1:ata2:0:1:0): CAM status: ATA Status Error
(ada1:ata2:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 40 (UNC )
(ada1:ata2:0:1:0): RES: 51 40 21 94 9c 02 02 00 00 00 00
(ada1:ata2:0:1:0): Error 5, Retries exhausted
dd: /dev/ada1b: Input/output error
I guess the disk is fucked, right?
Given that it's about 10 years old,
this is not surprising.
Anton
More information about the freebsd-current
mailing list