8.1 / strange bug during benchmark
Patrick Proniewski
patpro at patpro.net
Sat Oct 2 13:26:17 UTC 2010
Hi all,
I'm currently benchmarking my new server, testing UFS and ZFS, and
different settings of my RAID card.
I've used super-smack (from ports), to test MySQL performance
depending on FS and RAID card settings. After many runs, something in
the system becomes corrupted, and super-smack won't run unless I
reboot the server.
# uname -a
FreeBSD patbsd.univ-lyon2.fr 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon
Jul 19 02:36:49 UTC 2010 root at mason.cse.buffalo.edu:/usr/obj/usr/
src/sys/GENERIC amd64
# super-smack update-select.smack 2 10000
super-smack: Error creating semaphore errno = 28, error is No space
left on device
# df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/da0s1a 4.8G 266M 4.2G 6% /
devfs 1.0K 1.0K 0B 100% /dev
/dev/da0s1e 1.9G 20K 1.8G 0% /tmp
/dev/da0s1f 19G 2.3G 16G 13% /usr
/dev/da0s1d 16G 108M 15G 1% /var
home 866G 846M 865G 0% /usr/home (<- zfs)
Here is a partial truss output for super-smack, showing an ioctl error:
# truss super-smack update-select.smack 2 10000 2>&1| grep -C2 ERR#25
munmap(0x801079000,1601536) = 0 (0x0)
open("update-select.smack",O_RDONLY,0666) = 2 (0x2)
ioctl(2,TIOCGETA,0xffffdef0) ERR#25 'Inappropriate ioctl for device'
fstat(2,{ mode=-rw-r--r-- ,inode=120591,size=3527,blksize=16384 }) =
0 (0x0)
read(2,"// this is will be used in the t"...,16384) = 3527 (0xdc7)
--
stat("/etc/nsswitch.conf",{ mode=-rw-r--
r-- ,inode=588848,size=327,blksize=16384 }) = 0 (0x0)
open("/etc/nsswitch.conf",O_RDONLY,0666) = 3 (0x3)
ioctl(3,TIOCGETA,0xffffda20) ERR#25 'Inappropriate ioctl for device'
fstat(3,{ mode=-rw-r--r-- ,inode=588848,size=327,blksize=16384 }) = 0
(0x0)
read(3,"#\n# nsswitch.conf(5) - name ser"...,16384) = 327 (0x147)
--
access("/usr/lib/nss_dns.so.1",0) ERR#2 'No such file or directory'
sigprocmask(SIG_SETMASK,0x0,0x0) = 0 (0x0)
ioctl(3,TIOCGETA,0xffffda30) ERR#25 'Inappropriate ioctl for device'
close(3) = 0 (0x0)
sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|
SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|
SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|
SIGUSR2,0x0) = 0 (0x0)
--
close(3) = 0 (0x0)
fork() = 18889 (0x49c9)
ioctl(2,TIOCGETA,0xffffdf00) ERR#25 'Inappropriate ioctl for device'
wait4(0x49c9,0x0,0x0,0x0,0x4,0x1) = 18889 (0x49c9)
sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|
SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|
SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|
SIGUSR2,0x0) = 0 (0x0)
After a reboot, super-smack runs ok again, until it fails after a
large number of runs (I would say between 320 and 640 runs). The
system seems to behave normally apart from that.
Any idea what's going wrong?
regards,
patpro
-------------------------------------------------------------------------
dmesg output:
Copyright (c) 1992-2010 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 UTC 2010
root at mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(R) CPU E5450 @ 3.00GHz (3000.01-MHz K8-
class CPU)
Origin = "GenuineIntel" Id = 0x1067a Family = 6 Model = 17
Stepping = 10
Features
=
0xbfebfbff
<
FPU
,VME
,DE
,PSE
,TSC
,MSR
,PAE
,MCE
,CX8
,APIC
,SEP
,MTRR
,PGE
,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2
=
0x40ce3bd
<
SSE3
,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,XSAVE>
AMD Features=0x20100800<SYSCALL,NX,LM>
AMD Features2=0x1<LAHF>
TSC: P-state invariant
real memory = 17179869184 (16384 MB)
avail memory = 16562409472 (15795 MB)
ACPI APIC Table: <PTLTD APIC >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
cpu2 (AP): APIC ID: 2
cpu3 (AP): APIC ID: 3
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <PTLTD XSDT> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
unknown: I/O range not supported
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci2: <ACPI PCI bus> on pcib2
3ware device driver for 9000 series storage controllers, version:
3.70.05.001
twa0: <3ware 9000 series Storage Controller> port 0x2000-0x20ff mem
0xf8000000-0xf9ffffff,0xfc100000-0xfc100fff irq 16 at device 0.0 on pci2
twa0: [ITHREAD]
twa0: INFO: (0x15: 0x1300): Controller details:: Model 9650SE-4LPML, 4
ports, Firmware FE9X 3.08.00.016, BIOS BE9X 3.08.00.004
pcib3: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pci0: <base peripheral> at device 8.0 (no driver attached)
uhci0: <Intel 82801I (ICH9) USB controller> port 0x1820-0x183f irq 16
at device 26.0 on pci0
uhci0: [ITHREAD]
usbus0: <Intel 82801I (ICH9) USB controller> on uhci0
ehci0: <Intel 82801I (ICH9) USB 2.0 controller> mem
0xfc600000-0xfc6003ff irq 18 at device 26.7 on pci0
ehci0: [ITHREAD]
usbus1: EHCI version 1.0
usbus1: <Intel 82801I (ICH9) USB 2.0 controller> on ehci0
pcib4: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0
pci5: <ACPI PCI bus> on pcib5
em0: <Intel(R) PRO/1000 Network Connection 7.0.5> port 0x3000-0x301f
mem 0xfc200000-0xfc21ffff irq 16 at device 0.0 on pci5
em0: Using MSI interrupt
em0: [FILTER]
em0: Ethernet address: 00:30:48:d2:e3:fe
pcib6: <ACPI PCI-PCI bridge> irq 17 at device 28.5 on pci0
pci6: <ACPI PCI bus> on pcib6
em1: <Intel(R) PRO/1000 Network Connection 7.0.5> port 0x4000-0x401f
mem 0xfc300000-0xfc31ffff irq 17 at device 0.0 on pci6
em1: Using MSI interrupt
em1: [FILTER]
em1: Ethernet address: 00:30:48:d2:e3:ff
uhci1: <Intel 82801I (ICH9) USB controller> port 0x1840-0x185f irq 16
at device 29.0 on pci0
uhci1: [ITHREAD]
usbus2: <Intel 82801I (ICH9) USB controller> on uhci1
uhci2: <Intel 82801I (ICH9) USB controller> port 0x1860-0x187f irq 17
at device 29.1 on pci0
uhci2: [ITHREAD]
usbus3: <Intel 82801I (ICH9) USB controller> on uhci2
uhci3: <Intel 82801I (ICH9) USB controller> port 0x1880-0x189f irq 18
at device 29.2 on pci0
uhci3: [ITHREAD]
usbus4: <Intel 82801I (ICH9) USB controller> on uhci3
ehci1: <Intel 82801I (ICH9) USB 2.0 controller> mem
0xfc600400-0xfc6007ff irq 16 at device 29.7 on pci0
ehci1: [ITHREAD]
usbus5: EHCI version 1.0
usbus5: <Intel 82801I (ICH9) USB 2.0 controller> on ehci1
pcib7: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci7: <ACPI PCI bus> on pcib7
atapci0: <ITE IT8213F UDMA133 controller> port 0x5098-0x509f,
0x5090-0x5093,0x5080-0x508f irq 17 at device 0.0 on pci7
atapci0: [ITHREAD]
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
vgapci0: <VGA-compatible display> port 0x5000-0x507f mem
0xfa000000-0xfbffffff,0xfc000000-0xfc03ffff at device 1.0 on pci7
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci1: <Intel ICH9 SATA300 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x18d0-0x18df,0x18c0-0x18cf at
device 31.2 on pci0
ata0: <ATA channel 0> on atapci1
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci1
ata1: [ITHREAD]
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
atapci2: <Intel ICH9 SATA300 controller> port 0x1c28-0x1c2f,
0x1c1c-0x1c1f,0x1c20-0x1c27,0x1c18-0x1c1b,0x18f0-0x18ff,0x18e0-0x18ef
irq 18 at device 31.5 on pci0
atapci2: [ITHREAD]
ata3: <ATA channel 0> on atapci2
ata3: [ITHREAD]
ata4: <ATA channel 1> on atapci2
ata4: [ITHREAD]
acpi_button0: <Power Button> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart1: [FILTER]
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on
acpi0
fdc0: [FILTER]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc9fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on
isa0
ppc0: cannot reserve I/O port range
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 91e091e0600091e
device_attach: est0 attach returned 6
p4tcc0: <CPU Frequency Thermal Control> on cpu0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 91e091e0600091e
device_attach: est1 attach returned 6
p4tcc1: <CPU Frequency Thermal Control> on cpu1
est2: <Enhanced SpeedStep Frequency Control> on cpu2
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 91e091e0600091e
device_attach: est2 attach returned 6
p4tcc2: <CPU Frequency Thermal Control> on cpu2
est3: <Enhanced SpeedStep Frequency Control> on cpu3
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 91e091e0600091e
device_attach: est3 attach returned 6
p4tcc3: <CPU Frequency Thermal Control> on cpu3
Timecounters tick every 1.000 msec
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 480Mbps High Speed USB v2.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 12Mbps Full Speed USB v1.0
usbus5: 480Mbps High Speed USB v2.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
ugen4.1: <Intel> at usbus4
uhub4: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
ugen5.1: <Intel> at usbus5
uhub5: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus5
uhub0: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
uhub4: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub5: 6 ports with 6 removable, self powered
da0 at twa0 bus 0 scbus0 target 0 lun 0SMP: AP CPU #2 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
da0: <AMCC 9650SE-4LP DISK 3.08> Fixed Direct Access SCSI-5 device
da0: 100.000MB/s transfers
da0: 953664MB (1953103872 512 byte sectors: 255H 63S/T 121575C)
ugen2.2: <ATEN> at usbus2
ukbd0: <HID Keyboard> on usbus2
kbd2 at ukbd0
ums0: <HID Mouse> on usbus2
ums0: 5 buttons and [XYZ] coordinates ID=0
Trying to mount root from ufs:/dev/da0s1a
ZFS filesystem version 3
ZFS storage pool version 14
em0: link state changed to UP
More information about the freebsd-testing
mailing list