8-STABLE performance issues on Supermicro Core i7

Case van Rij case.vanrij at gmail.com
Tue May 4 05:45:40 UTC 2010


I was a little surprised to read this, I've been running 7.1 based
code on some supermicro nehalem machines (5500, Xeon, X8DT6/X8DTE)
which perform reasonably well, and was hoping to test some X8SAX based
systems with core i7 to compare performance.

The configuration Ive been using so far:
2x E5520 Xeon, SMT disabled ie. 2 x 4 cores
24GB (3x 4 GB per CPU socket)
LSILOGIC SASX36 A1 7015, SEAGATE ST9300603SS 0006

This made me curious enough to put 8.0-RELEASE on one of these to find
out how it would do on buildkernel, and it seems to perform 'as
expected',

nhm005# /usr/bin/time -h make buildkernel
        18m20.31s real          13m13.84s user          5m1.47s sys

nhm005# /usr/bin/time -h make -j4 buildkernel
        8m53.96s real           13m22.34s user          6m54.47s sys

A misbehaving mpt controller/driver seemed to slow this down a bit, it
was logging full queue events during the initial install and continued
to do so during buildkernel.

mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 127
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128

I think it's safe to say the x8sax tests just moved up on my TODO list
a little, unfortunately that hardware is tied up at the moment.

I'd be interested to see lmbench results if you have any you want to
share. (/usr/ports/benchmarks/lmbench)
I'll attach dmesg & lmbench results for the X8DT6/E5520 configuration.

Case
-------------- next part --------------
Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:02:08 UTC 2009
    root at mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz (2270.05-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x106a5  Stepping = 5
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x9ce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant
real memory  = 25769803776 (24576 MB)
avail memory = 24821891072 (23672 MB)
ACPI APIC Table: <092409 APIC1926>
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 2 package(s) x 4 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  2
 cpu2 (AP): APIC ID:  4
 cpu3 (AP): APIC ID:  6
 cpu4 (AP): APIC ID: 16
 cpu5 (AP): APIC ID: 18
 cpu6 (AP): APIC ID: 20
 cpu7 (AP): APIC ID: 22
ioapic0: Changing APIC ID to 1
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <NEC > on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, 9f800000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci9: <ACPI PCI bus> on pcib1
pci9: <network, ethernet> at device 0.0 (no driver attached)
pcib2: <ACPI PCI-PCI bridge> at device 3.0 on pci0
pci8: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 5.0 on pci0
pci7: <ACPI PCI bus> on pcib3
pci7: <serial bus> at device 0.0 (no driver attached)
pcib4: <ACPI PCI-PCI bridge> at device 7.0 on pci0
pci6: <ACPI PCI bus> on pcib4
mpt0: <LSILogic SAS/SATA Adapter> port 0xe000-0xe0ff mem 0xfadfc000-0xfadfffff,0xfade0000-0xfadeffff irq 16 at device 0.0 on pci6
mpt0: [ITHREAD]
mpt0: MPI Version=1.5.19.0
mpt0: Capabilities: ( RAID-0 RAID-1E RAID-1 )
mpt0: 0 Active Volumes (2 Max)
mpt0: 0 Hidden Drive Members (14 Max)
pcib5: <ACPI PCI-PCI bridge> at device 9.0 on pci0
pci5: <ACPI PCI bus> on pcib5
pci5: <mass storage, SAS> at device 0.0 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 20.0 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 20.1 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 20.2 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 20.3 (no driver attached)
pci0: <base peripheral> at device 22.0 (no driver attached)
pci0: <base peripheral> at device 22.1 (no driver attached)
pci0: <base peripheral> at device 22.2 (no driver attached)
pci0: <base peripheral> at device 22.3 (no driver attached)
pci0: <base peripheral> at device 22.4 (no driver attached)
pci0: <base peripheral> at device 22.5 (no driver attached)
pci0: <base peripheral> at device 22.6 (no driver attached)
pci0: <base peripheral> at device 22.7 (no driver attached)
uhci0: <UHCI (generic) USB controller> port 0x8800-0x881f irq 16 at device 26.0 on pci0
uhci0: [ITHREAD]
uhci0: LegSup = 0x0f30
usbus0: <UHCI (generic) USB controller> on uhci0
uhci1: <UHCI (generic) USB controller> port 0x8480-0x849f irq 21 at device 26.1 on pci0
uhci1: [ITHREAD]
uhci1: LegSup = 0x0f30
usbus1: <UHCI (generic) USB controller> on uhci1
uhci2: <UHCI (generic) USB controller> port 0x8400-0x841f irq 19 at device 26.2 on pci0
uhci2: [ITHREAD]
uhci2: LegSup = 0x0f30
usbus2: <UHCI (generic) USB controller> on uhci2
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xf97fa000-0xf97fa3ff irq 18 at device 26.7 on pci0
ehci0: [ITHREAD]
usbus3: EHCI version 1.0
usbus3: <EHCI (generic) USB 2.0 controller> on ehci0
pcib6: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib6
pcib7: <ACPI PCI-PCI bridge> irq 17 at device 28.4 on pci0
pci3: <ACPI PCI bus> on pcib7
em0: <Intel(R) PRO/1000 Network Connection 6.9.14> port 0xbc00-0xbc1f mem 0xfa7e0000-0xfa7fffff,0xfa7dc000-0xfa7dffff irq 16 at device 0.0 on pci3
em0: Using MSIX interrupts
em0: [ITHREAD]
em0: [ITHREAD]
em0: [ITHREAD]
em0: Ethernet address: 00:30:48:dd:f2:3c
pcib8: <ACPI PCI-PCI bridge> irq 16 at device 28.5 on pci0
pci4: <ACPI PCI bus> on pcib8
em1: <Intel(R) PRO/1000 Network Connection 6.9.14> port 0xcc00-0xcc1f mem 0xfa8e0000-0xfa8fffff,0xfa8dc000-0xfa8dffff irq 17 at device 0.0 on pci4
em1: Using MSIX interrupts
em1: [ITHREAD]
em1: [ITHREAD]
em1: [ITHREAD]
em1: Ethernet address: 00:30:48:dd:f2:3d
uhci3: <UHCI (generic) USB controller> port 0x9000-0x901f irq 23 at device 29.0 on pci0
uhci3: [ITHREAD]
uhci3: LegSup = 0x0f30
usbus4: <UHCI (generic) USB controller> on uhci3
uhci4: <UHCI (generic) USB controller> port 0x8c00-0x8c1f irq 19 at device 29.1 on pci0
uhci4: [ITHREAD]
uhci4: LegSup = 0x0f30
usbus5: <UHCI (generic) USB controller> on uhci4
uhci5: <UHCI (generic) USB controller> port 0x8880-0x889f irq 18 at device 29.2 on pci0
uhci5: [ITHREAD]
uhci5: LegSup = 0x0f30
usbus6: <UHCI (generic) USB controller> on uhci5
ehci1: <EHCI (generic) USB 2.0 controller> mem 0xf97fc000-0xf97fc3ff irq 23 at device 29.7 on pci0
ehci1: [ITHREAD]
usbus7: EHCI version 1.0
usbus7: <EHCI (generic) USB 2.0 controller> on ehci1
pcib9: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci1: <ACPI PCI bus> on pcib9
vgapci0: <VGA-compatible display> mem 0xf8000000-0xf8ffffff,0xfa6fc000-0xfa6fffff,0xf9800000-0xf9ffffff irq 16 at device 3.0 on pci1
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH10 SATA300 controller> port 0xac00-0xac07,0xa880-0xa883,0xa800-0xa807,0xa480-0xa483,0xa400-0xa40f,0xa080-0xa08f irq 19 at device 31.2 on pci0
atapci0: [ITHREAD]
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0
ata3: [ITHREAD]
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
atapci1: <Intel ICH10 SATA300 controller> port 0x9c00-0x9c07,0x9880-0x9883,0x9800-0x9807,0x9480-0x9483,0x9400-0x940f,0x9080-0x908f irq 19 at device 31.5 on pci0
atapci1: [ITHREAD]
ata4: <ATA channel 0> on atapci1
ata4: [ITHREAD]
ata5: <ATA channel 1> on atapci1
ata5: [ITHREAD]
acpi_button0: <Power Button> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
uart0: console (9600,n,8,1)
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart1: [FILTER]
ACPI Warning: \\_SB_.PCI0.SBRG.FDC_._FDE: Return type mismatch - found Package, expected Buffer 20090521 nspredef-1051
fdc0: <floppy drive controller (FDE)> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: [FILTER]
uart2: <16550 or compatible> port 0x3e8-0x3ef irq 5 drq 2 on acpi0
uart2: [FILTER]
cpu0: <ACPI CPU> on acpi0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 11
device_attach: est0 attach returned 6
p4tcc0: <CPU Frequency Thermal Control> on cpu0
cpu1: <ACPI CPU> on acpi0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 11
device_attach: est1 attach returned 6
p4tcc1: <CPU Frequency Thermal Control> on cpu1
cpu2: <ACPI CPU> on acpi0
est2: <Enhanced SpeedStep Frequency Control> on cpu2
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 11
device_attach: est2 attach returned 6
p4tcc2: <CPU Frequency Thermal Control> on cpu2
cpu3: <ACPI CPU> on acpi0
est3: <Enhanced SpeedStep Frequency Control> on cpu3
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 11
device_attach: est3 attach returned 6
p4tcc3: <CPU Frequency Thermal Control> on cpu3
cpu4: <ACPI CPU> on acpi0
est4: <Enhanced SpeedStep Frequency Control> on cpu4
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 11
device_attach: est4 attach returned 6
p4tcc4: <CPU Frequency Thermal Control> on cpu4
cpu5: <ACPI CPU> on acpi0
est5: <Enhanced SpeedStep Frequency Control> on cpu5
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 11
device_attach: est5 attach returned 6
p4tcc5: <CPU Frequency Thermal Control> on cpu5
cpu6: <ACPI CPU> on acpi0
est6: <Enhanced SpeedStep Frequency Control> on cpu6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 11
device_attach: est6 attach returned 6
p4tcc6: <CPU Frequency Thermal Control> on cpu6
cpu7: <ACPI CPU> on acpi0
est7: <Enhanced SpeedStep Frequency Control> on cpu7
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 11
device_attach: est7 attach returned 6
p4tcc7: <CPU Frequency Thermal Control> on cpu7
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xc97ff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x100>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
ppc0: cannot reserve I/O port range
Timecounters tick every 1.000 msec
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 480Mbps High Speed USB v2.0
usbus4: 12Mbps Full Speed USB v1.0
usbus5: 12Mbps Full Speed USB v1.0
usbus6: 12Mbps Full Speed USB v1.0
usbus7: 480Mbps High Speed USB v2.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
ugen4.1: <Intel> at usbus4
uhub4: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
ugen5.1: <Intel> at usbus5
uhub5: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5
ugen6.1: <Intel> at usbus6
uhub6: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6
ugen7.1: <Intel> at usbus7
uhub7: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus7
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub4: 2 ports with 2 removable, self powered
uhub5: 2 ports with 2 removable, self powered
uhub6: 2 ports with 2 removable, self powered
uhub3: 6 ports with 6 removable, self powered
uhub7: 6 ports with 6 removable, self powered
ses0 at mpt0 bus 0 target 24 lun 0
ses0: <LSILOGIC SASX36 A1 7015> Fixed Enclosure Services SCSI-3 device 
ses0: 300.000MB/s transfers
ses0: Command Queueing enabled
ses0: SCSI-3 SES Device
da0 at mpt0 bus 0 target 0 lun 0
da0: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da2 at mpt0 bus 0 target 2 lun 0
da2: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da2: 300.000MB/s transfers
da2: Command Queueing enabled
da2: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da3 at mpt0 bus 0 target 3 lun 0
da3: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da3: 300.000MB/s transfers
da3: Command Queueing enabled
da3: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da1 at mpt0 bus 0 target 1 lun 0
da1: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da1: 300.000MB/s transfers
da1: Command Queueing enabled
da1: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
SMP: AP CPU #3 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #5 Launched!
da4 at mpt0 bus 0 target 4 lun 0
da4: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da4: 300.000MB/s transfers
da4: Command Queueing enabled
da4: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da5 at mpt0 bus 0 target 5 lun 0
da5: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da5: 300.000MB/s transfers
da5: Command Queueing enabled
da5: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da6 at mpt0 bus 0 target 6 lun 0
da6: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da6: 300.000MB/s transfers
da6: Command Queueing enabled
da6: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da7 at mpt0 bus 0 target 7 lun 0
da7: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da7: 300.000MB/s transfers
da7: Command Queueing enabled
da7: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da8 at mpt0 bus 0 target 9 lun 0
da8: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da8: 300.000MB/s transfers
da8: Command Queueing enabled
da8: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da9 at mpt0 bus 0 target 10 lun 0
da9: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da9: 300.000MB/s transfers
da9: Command Queueing enabled
da9: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da10 at mpt0 bus 0 target 11 lun 0
da10: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da10: 300.000MB/s transfers
da10: Command Queueing enabled
da10: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da14 at mpt0 bus 0 target 15 lun 0
da14: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da14: 300.000MB/s transfers
da14: Command Queueing enabled
da14: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da11 at mpt0 bus 0 target 12 lun 0
da11: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da11: 300.000MB/s transfers
da11: Command Queueing enabled
da11: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da12 at mpt0 bus 0 target 13 lun 0
da12: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da12: 300.000MB/s transfers
da12: Command Queueing enabled
da12: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da13 at mpt0 bus 0 target 14 lun 0
da13: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da13: 300.000MB/s transfers
da13: Command Queueing enabled
da13: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da15 at mpt0 bus 0 target 16 lun 0
da15: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da15: 300.000MB/s transfers
da15: Command Queueing enabled
da15: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da16 at mpt0 bus 0 target 17 lun 0
da16: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da16: 300.000MB/s transfers
da16: Command Queueing enabled
da16: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da17 at mpt0 bus 0 target 18 lun 0
da17: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da17: 300.000MB/s transfers
da17: Command Queueing enabled
da17: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da18 at mpt0 bus 0 target 19 lun 0
da18: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da18: 300.000MB/s transfers
da18: Command Queueing enabled
da18: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da19 at mpt0 bus 0 target 20 lun 0
da19: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da19: 300.000MB/s transfers
da19: Command Queueing enabled
da19: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da20 at mpt0 bus 0 target 21 lun 0
da20: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da20: 300.000MB/s transfers
da20: Command Queueing enabled
da20: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da21 at mpt0 bus 0 target 22 lun 0
da21: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da21: 300.000MB/s transfers
da21: Command Queueing enabled
da21: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da22 at mpt0 bus 0 target 23 lun 0
da22: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da22: 300.000MB/s transfers
da22: Command Queueing enabled
da22: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da23 at mpt0 bus 0 target 26 lun 0
da23: <SEAGATE ST9300603SS 0006> Fixed Direct Access SCSI-5 device 
da23: 300.000MB/s transfers
da23: Command Queueing enabled
da23: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
GEOM: da6: partition 1 does not start on a track boundary.
GEOM: da6: partition 1 does not end on a track boundary.
Trying to mount root from ufs:/dev/da0s1a
ugen2.2: <American Megatrends Inc.> at usbus2
ukbd0: <Keyboard Interface> on usbus2
kbd2 at ukbd0
ums0: <Mouse Interface> on usbus2
em0: link state changed to UP
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 127
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 129
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 127
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 127
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 129
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x00 Depth 128
-------------- next part --------------
[lmbench3.0 results for FreeBSD nhm005.west.isilon.com 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:02:08 UTC 2009 root at mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64]
[LMBENCH_VER: 3.0-a9]
[BENCHMARK_HARDWARE: YES]
[BENCHMARK_OS: YES]
[ALL: 512 1k 2k 4k 8k 16k 32k 64k 128k 256k 512k 1m 2m 4m 8m 16m 32m 64m 128m 256m 512m 1024m 2048m]
[DISKS: /dev/da1 ]
[DISK_DESC: [/dev/da1:ST9300603SS] ]
[ENOUGH: 5000]
[FAST: ]
[FASTMEM: NO]
[FILE: /var/tmp/XXX]
[FSDIR: /var/tmp]
[HALF: 512 1k 2k 4k 8k 16k 32k 64k 128k 256k 512k 1m 2m 4m 8m 16m 32m 64m 128m 256m 512m 1024m]
[INFO: INFO.nhm005.west.isilon.com]
[LINE_SIZE: ]
[LOOP_O: 0.00000041]
[MB: 2867]
[MHZ: 2256 MHz, 0.4433 nanosec clock]
[MOTHERBOARD: ]
[NETWORKS: ]
[PROCESSORS: ]
[REMOTE: ]
[SLOWFS: NO]
[OS: amd64-freebsd8.0]
[SYNC_MAX: 1]
[LMBENCH_SCHED: DEFAULT]
[TIMING_O: 1]
[LMBENCH VERSION: 3.0-a9]
[USER: root]
[HOSTNAME: nhm005.west.isilon.com]
[NODENAME: nhm005.west.isilon.com]
[SYSNAME: FreeBSD]
[PROCESSOR: amd64]
[MACHINE: amd64]
[RELEASE: 8.0-RELEASE]
[VERSION: FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:02:08 UTC 2009 root at mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC ]
[Mon May 3 21:33:12 UTC 2010]
[ 9:33PM up 1:05, 3 users, load averages: 0.00, 0.07, 0.31]
[net: Name    Mtu Network       Address              Ipkts Ierrs    Opkts Oerrs  Coll]
[net: em0    1500 <Link#1>      00:30:48:dd:f2:3c   246559     0    64557     0     0]
[if: em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500]
[if: options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>]
[if: ether 00:30:48:dd:f2:3c]
[if: inet 10.53.193.164 netmask 0xffff0000 broadcast 10.53.255.255]
[if: media: Ethernet autoselect (1000baseT <full-duplex>)]
[if: status: active]
[net: em0    1500 10.53.0.0     nhm005               78799     -    56165     -     -]
[if: em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500]
[if: options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>]
[if: ether 00:30:48:dd:f2:3c]
[if: inet 10.53.193.164 netmask 0xffff0000 broadcast 10.53.255.255]
[if: media: Ethernet autoselect (1000baseT <full-duplex>)]
[if: status: active]
[net: em1*   1500 <Link#2>      00:30:48:dd:f2:3d        0     0        0     0     0]
ifconfig: interface em1* does not exist
[net: lo0   16384 <Link#3>                               2     0        2     0     0]
[if: lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384]
[if: options=3<RXCSUM,TXCSUM>]
[if: inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3]
[if: inet6 ::1 prefixlen 128]
[if: inet 127.0.0.1 netmask 0xff000000]
[net: lo0   16384 fe80:3::1     fe80:3::1                0     -        0     -     -]
[if: lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384]
[if: options=3<RXCSUM,TXCSUM>]
[if: inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3]
[if: inet6 ::1 prefixlen 128]
[if: inet 127.0.0.1 netmask 0xff000000]
[net: lo0   16384 localhost     ::1                      0     -        0     -     -]
[if: lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384]
[if: options=3<RXCSUM,TXCSUM>]
[if: inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3]
[if: inet6 ::1 prefixlen 128]
[if: inet 127.0.0.1 netmask 0xff000000]
[net: lo0   16384 your-net      localhost                2     -        2     -     -]
[if: lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384]
[if: options=3<RXCSUM,TXCSUM>]
[if: inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3]
[if: inet6 ::1 prefixlen 128]
[if: inet 127.0.0.1 netmask 0xff000000]
[mount: /dev/da0s1a on / (ufs, local)]
[mount: devfs on /dev (devfs, local, multilabel)]
[mount: /dev/da0s1e on /tmp (ufs, local, soft-updates)]
[mount: /dev/da0s1f on /usr (ufs, local, soft-updates)]
[mount: /dev/da0s1d on /var (ufs, local, soft-updates)]
Simple syscall: 0.2310 microseconds
Simple read: 0.3454 microseconds
Simple write: 0.3303 microseconds
Simple stat: 3.2043 microseconds
Simple fstat: 0.4714 microseconds
Simple open/close: 4.6664 microseconds
Select on 10 fd's: 0.7659 microseconds
Select on 100 fd's: 14.3446 microseconds
Select on 250 fd's: 35.9935 microseconds
Select on 500 fd's: 74.4189 microseconds
Select on 10 tcp fd's: 0.6291 microseconds
Select on 100 tcp fd's: 8.1481 microseconds
Select on 250 tcp fd's: 20.7689 microseconds
Select on 500 tcp fd's: 41.8092 microseconds
Signal handler installation: 0.3163 microseconds
Signal handler overhead: 1.4245 microseconds
Protection fault: 0.1793 microseconds
Pipe latency: 9.4332 microseconds
AF_UNIX sock stream latency: 12.7726 microseconds
Process fork+exit: 830.5714 microseconds
Process fork+execve: 2198.0000 microseconds
Process fork+/bin/sh -c: 4528.5000 microseconds
integer bit: 0.45 nanoseconds
integer add: 0.22 nanoseconds
integer mul: 0.15 nanoseconds
integer div: 10.64 nanoseconds
integer mod: 10.16 nanoseconds
int64 bit: 0.44 nanoseconds
uint64 add: 0.22 nanoseconds
int64 mul: 0.15 nanoseconds
int64 div: 19.70 nanoseconds
int64 mod: 19.46 nanoseconds
float add: 1.33 nanoseconds
float mul: 1.78 nanoseconds
float div: 6.58 nanoseconds
double add: 1.33 nanoseconds
double mul: 2.22 nanoseconds
double div: 10.13 nanoseconds
float bogomflops: 6.20 nanoseconds
double bogomflops: 9.74 nanoseconds
integer bit parallelism: 2.94
integer add parallelism: 1.84
integer mul parallelism: 3.26
integer div parallelism: 1.92
integer mod parallelism: 2.17
int64 bit parallelism: 2.54
int64 add parallelism: 1.88
int64 mul parallelism: 2.72
int64 div parallelism: 1.53
int64 mod parallelism: 1.47
float add parallelism: 3.00
float mul parallelism: 4.05
float div parallelism: 1.06
double add parallelism: 3.00
double mul parallelism: 5.05
double div parallelism: 1.04
File /var/tmp/XXX write bandwidth: 118113 KB/sec
lat_pagefault: /var/tmp/XXX too small
lat_pagefault: /var/tmp/XXX too small

"mappings
0.524288 81
1.048576 127
2.097152 247
4.194304 454
8.388608 979
16.777216 1987
33.554432 3886
67.108864 6348
134.217728 12550
268.435456 27186
536.870912 51699
1073.741824 87608
2147.483648 211458

"File system latency
0k	366	62108	43416
1k	169	30157	26638
4k	154	28986	18855
10k	124	21835	19946

"Seek times for /dev/da1
299699.7 19.23
299849.7 18.58
299399.7 15.63
299549.7 19.26
299099.7 15.69
299249.7 18.94
298799.7 14.12
298949.7 18.22
298499.7 18.32
298649.7 14.88
298199.7 18.29
298349.7 16.84
297899.7 17.24
298049.7 16.88
297599.7 19.07
297749.7 18.49
297299.7 19.33
297449.7 19.63
296999.7 16.85
297149.7 15.22
296699.7 18.58
296849.7 19.42
296399.7 14.11
296549.7 15.68
296099.7 15.25
296249.7 18.49
295799.7 18.58
295949.7 14.63
295499.7 15.79
295649.7 17.89
295199.7 16.08
295349.7 16.77
294899.7 16.35
295049.7 17.73
294599.7 14.44
294749.7 18.76
294299.7 17.02
294449.7 19.62
293999.7 16.92
294149.7 14.00
293699.7 14.60
293849.7 15.96
293399.7 16.76
293549.7 18.65
293099.7 14.23
293249.7 18.74
292799.7 16.10
292949.7 14.66
292499.7 15.88
292649.7 17.31
292199.7 17.69
292349.7 19.14
291899.7 19.20
292049.7 14.01
291599.7 19.29
291749.7 15.70
291299.7 19.02
291449.7 17.91
290999.7 18.58
291149.7 18.22
290699.7 16.94
290849.7 19.66
290399.7 18.28
290549.7 14.95
290099.7 18.61
290249.7 15.64
289799.7 18.67
289949.7 13.76
289499.7 16.78
289649.7 17.32
289199.7 18.48
289349.7 14.72
288899.7 16.75
289049.7 16.80
288599.7 17.29
288749.7 18.31
288299.7 15.31
288449.7 16.57
287999.7 13.96
288149.7 18.30
287699.7 16.13
287849.7 17.08
287399.7 15.77
287549.7 15.72
287099.7 18.77
287249.7 14.98
286799.7 17.64
286949.7 17.89
286499.7 18.66
286649.7 14.30
286199.7 15.40
286349.7 16.73
285899.7 15.54
286049.7 19.21
285599.7 15.92
285749.7 18.64
285299.7 19.36
285449.7 18.75
284999.7 17.79
285149.7 15.41
284699.7 14.61
284849.7 14.13
284399.7 17.98
284549.7 17.12
284099.7 18.16
284249.7 19.22
283799.7 18.29
283949.7 14.98
283499.7 17.84
283649.7 18.20
283199.7 16.71
283349.7 16.72
282899.7 19.33
283049.7 17.99
282599.7 16.41
282749.7 16.80
282299.7 18.55
282449.7 19.32
281999.7 19.56
282149.7 18.90
281699.7 15.66
281849.7 14.28
281399.7 17.34
281549.7 15.69
281099.7 17.59
281249.7 15.62
280799.7 16.45
280949.7 17.92
280499.7 16.98
280649.7 15.35
280199.7 14.85
280349.7 19.21
279899.7 16.27
280049.7 16.70
279599.7 16.18
279749.7 15.17
279299.7 19.02
279449.7 17.43
278999.7 13.66
279149.7 17.62
278699.7 15.75
278849.7 18.78
278399.7 17.29
278549.7 15.91
278099.7 16.96
278249.7 14.07
277799.7 19.52
277949.7 13.69
277499.7 19.20
277649.7 17.15
277199.7 15.11
277349.7 18.09
276899.7 13.74
277049.7 16.17
276599.7 13.98
276749.7 16.26
276299.7 18.73
276449.7 18.51
275999.7 17.94
276149.7 18.20
275699.7 14.44
275849.7 18.76
275399.7 14.92
275549.7 19.33
275099.7 18.73
275249.7 14.38
274799.7 17.75
274949.7 15.42
274499.7 15.62
274649.7 17.58
274199.7 14.02
274349.7 14.23
273899.7 18.67
274049.7 16.72
273599.7 17.35
273749.7 14.99
273299.7 15.94
273449.7 15.38
272999.7 16.65
273149.7 16.56
272699.7 14.17
272849.7 17.57
272399.7 16.49
272549.7 17.71
272099.7 19.40
272249.7 15.49
271799.7 17.38
271949.7 15.81
271499.7 16.96
271649.7 15.86
271199.7 17.19
271349.7 16.87
270899.7 14.32
271049.7 13.99
270599.7 17.72
270749.7 14.38
270299.7 15.26
270449.7 17.95
269999.7 14.12
270149.7 15.10
269699.7 17.40
269849.7 17.89
269399.7 14.18
269549.7 16.74
269099.7 15.09
269249.7 18.11
268799.7 13.90
268949.7 16.60
268499.7 15.91
268649.7 17.06
268199.7 15.01
268349.7 16.06
267899.7 16.00
268049.7 17.20
267599.7 14.60
267749.7 11.51
267299.7 14.99
267449.7 17.47
266999.7 14.59
267149.7 16.03
266699.7 16.04
266849.7 16.30
266399.7 15.76
266549.7 17.20
266099.7 15.30
266249.7 14.01
265799.7 17.82
265949.7 14.06
265499.7 18.01
265649.7 15.58
265199.7 16.39
265349.7 16.81
264899.7 15.69
265049.7 16.36
264599.7 15.71
264749.7 14.05
264299.7 18.01
264449.7 13.70
263999.7 18.12
264149.7 15.64
263699.7 16.86
263849.7 16.34
263399.7 15.72
263549.7 15.53
263099.7 16.53
263249.7 16.65
262799.7 15.42
262949.7 14.91
262499.7 17.30
262649.7 15.90
262199.7 16.17
262349.7 14.22
261899.7 17.85
262049.7 18.17
261599.7 13.90
261749.7 17.81
261299.7 14.65
261449.7 14.54
260999.7 17.52
261149.7 15.68
260699.7 16.15
260849.7 18.37
260399.7 13.69
260549.7 13.88
260099.7 18.62
260249.7 14.08
259799.7 18.00
259949.7 14.97
259499.7 17.10
259649.7 18.10
259199.7 13.96
259349.7 14.95
258899.7 17.32
259049.7 15.46
258599.7 16.59
258749.7 14.27
258299.7 17.80
258449.7 15.40
257999.7 16.66
258149.7 15.32
257699.7 17.18
257849.7 11.47
257399.7 14.35
257549.7 13.54
257099.7 18.53
257249.7 14.74
256799.7 17.32
256949.7 16.19
256499.7 16.31
256649.7 17.30
256199.7 14.76
256349.7 13.54
255899.7 18.52
256049.7 13.58
255599.7 18.25
255749.7 14.71
255299.7 17.79
255449.7 11.67
254999.7 8.39
255149.7 9.65
254699.7 10.41
254849.7 7.58
254399.7 18.48
254549.7 14.06
254099.7 18.44
254249.7 13.63
253799.7 18.19
253949.7 14.47
253499.7 17.59
253649.7 13.23
253199.7 18.83
253349.7 14.13
252899.7 18.37
253049.7 13.73
252599.7 18.34
252749.7 14.36
252299.7 17.47
252449.7 7.60
251999.6 18.46
252149.6 13.20
251699.6 13.24
251849.6 13.97
251399.6 18.09
251549.6 17.65
251099.6 14.41
251249.6 11.03
250799.6 15.04
250949.6 18.30
250499.6 13.96
250649.6 13.24
250199.6 18.79
250349.6 13.94
249899.6 18.13
250049.6 11.44
249599.6 14.62
249749.6 11.59
249299.6 14.92
249449.6 18.18
248999.6 13.60
249149.6 13.60
248699.6 18.47
248849.6 12.85
248399.6 13.21
248549.6 13.47
248099.6 13.69
248249.6 17.03
247799.6 15.03
247949.6 18.51
247499.6 13.56
247649.6 10.37
247199.6 9.90
247349.6 10.96
246899.6 14.98
247049.6 15.17
246599.6 16.89
246749.6 16.39
246299.6 15.68
246449.6 13.52
245999.6 12.99
246149.6 9.50
245699.6 16.57
245849.6 13.35
245399.6 18.48
245549.6 13.96
245099.6 17.56
245249.6 15.98
244799.6 14.20
244949.6 9.73
244499.6 14.95
244649.6 13.85
244199.6 14.14
244349.6 8.49
243899.6 13.97
244049.6 13.55
243599.6 11.56
243749.6 12.97
243299.6 16.09
243449.6 16.16
242999.6 15.91
243149.6 14.25
242699.6 16.86
242849.6 16.43
242399.6 12.87
242549.6 18.41
242099.6 16.22
242249.6 10.17
241799.6 18.56
241949.6 11.15
241499.6 18.38
241649.6 14.24
241199.6 16.34
241349.6 11.19
240899.6 11.51
241049.6 12.40
240599.6 8.31
240749.6 10.85
240299.6 11.76
240449.6 8.54
239999.6 17.02
240149.6 10.50
239699.6 15.51
239849.6 17.21
239399.6 17.64
239549.6 18.51
239099.6 13.71
239249.6 16.44
238799.6 15.63
238949.6 17.89
238499.6 17.14
238649.6 16.00
238199.6 15.02
238349.6 15.26
237899.6 17.05
238049.6 14.98
237599.6 13.74
237749.6 16.55
237299.6 14.41
237449.6 13.11
236999.6 13.05
237149.6 16.86
236699.6 14.67
236849.6 18.56
236399.6 16.43
236549.6 13.78
236099.6 13.26
236249.6 14.26
235799.6 13.11
235949.6 15.85
235499.6 16.21
235649.6 11.60
235199.6 15.19
235349.6 14.96
234899.6 18.41
235049.6 15.11
234599.6 12.95
234749.6 16.53
234299.6 15.28
234449.6 12.68
233999.6 14.49
234149.6 13.65
233699.6 16.18
233849.6 18.02
233399.6 16.71
233549.6 10.81
233099.6 16.43
233249.6 7.48
232799.6 17.06
232949.6 18.27
232499.6 17.59
232649.6 16.75
232199.6 15.27
232349.6 17.35
231899.6 14.29
232049.6 17.43
231599.6 17.71
231749.6 8.00
231299.6 18.05
231449.6 18.10
230999.6 12.87
231149.6 14.64
230699.6 8.79
230849.6 16.78
230399.6 14.74
230549.6 11.12
230099.6 11.84
230249.6 12.63
229799.6 12.62
229949.6 13.24
229499.6 15.38
229649.6 18.13
229199.6 16.60
229349.6 7.32
228899.6 10.32
229049.6 9.67
228599.6 12.82
228749.6 12.86
228299.6 12.81
228449.6 13.80
227999.6 18.27
228149.6 9.56
227699.6 13.16
227849.6 16.69
227399.6 14.99
227549.6 14.93
227099.6 12.64
227249.6 14.88
226799.6 13.20
226949.6 14.71
226499.6 10.55
226649.6 11.88
226199.6 11.63
226349.6 10.10
225899.6 12.62
226049.6 17.15
225599.6 16.63
225749.6 16.89
225299.6 15.34
225449.6 8.63
224999.6 14.20
225149.6 7.55
224699.6 17.62
224849.6 11.65
224399.6 9.91
224549.6 11.36
224099.6 10.15
224249.6 8.00
223799.6 12.06
223949.6 15.46
223499.6 16.93
223649.6 12.99
223199.6 8.40
223349.6 13.12
222899.6 18.18
223049.6 17.82
222599.6 16.91
222749.6 9.43
222299.6 11.67
222449.6 12.09
221999.6 14.68
222149.6 12.98
221699.6 9.98
221849.6 11.54
221399.6 6.51
221549.6 11.47
221099.6 12.89
221249.6 17.19
220799.6 12.79
220949.6 8.13
220499.6 11.93
220649.6 9.35
220199.6 8.97
220349.6 17.27
219899.6 13.14
220049.6 18.30
219599.6 13.77
219749.6 16.14
219299.6 14.60
219449.6 12.91
218999.6 10.64
219149.6 9.46
218699.6 13.31
218849.6 10.93
218399.6 11.80
218549.6 8.29
218099.6 7.92
218249.6 11.42
217799.6 16.75
217949.6 16.77
217499.6 9.52
217649.6 8.62
217199.6 13.81
217349.6 18.13
216899.6 14.83
217049.6 15.99
216599.6 18.07
216749.6 15.21
216299.6 14.83
216449.6 13.75
215999.6 13.16
216149.6 9.94
215699.6 17.88
215849.6 12.03
215399.6 12.25
215549.6 15.27
215099.6 17.69
215249.6 14.72
214799.6 13.10
214949.6 10.84
214499.6 13.44
214649.6 8.75
214199.6 11.87
214349.6 11.37
213899.6 16.68
214049.6 16.83
213599.6 13.45
213749.6 10.71
213299.6 10.02
213449.6 10.60
212999.6 17.40
213149.6 12.75
212699.6 17.29
212849.6 10.23
212399.6 16.73
212549.6 7.70
212099.6 6.85
212249.6 8.14
211799.6 11.35
211949.6 6.57
211499.6 8.39
211649.6 7.12
211199.6 6.82
211349.6 7.60
210899.6 12.55
211049.6 11.36
210599.6 9.37
210749.6 9.15
210299.6 10.35
210449.6 10.58
209999.6 10.25
210149.6 11.27
209699.6 9.46
209849.6 8.70
209399.6 9.91
209549.6 10.71
209099.6 17.75
209249.6 6.40
208799.6 7.59
208949.6 8.06
208499.6 9.88
208649.6 10.64
208199.6 14.95
208349.6 9.96
207899.6 13.57
208049.6 10.35
207599.6 6.94
207749.6 7.47
207299.6 15.24
207449.6 12.27
206999.6 8.79
207149.6 9.13
206699.6 7.49
206849.6 7.13
206399.6 12.73
206549.6 6.32
206099.6 12.03
206249.6 9.25
205799.6 6.69
205949.6 11.46
205499.6 11.78
205649.6 8.84
205199.6 13.03
205349.6 17.15
204899.6 10.14
205049.6 11.38
204599.6 9.23
204749.6 7.62
204299.6 12.07
204449.6 10.51
203999.6 9.58
204149.6 8.61
203699.6 9.14
203849.6 11.00
203399.6 8.68
203549.6 6.83
203099.6 9.83
203249.6 8.08
202799.6 7.04
202949.6 9.66
202499.6 10.26
202649.6 6.70
202199.6 12.64
202349.6 8.63
201899.6 9.53
202049.6 8.91
201599.6 11.01
201749.6 9.55
201299.6 6.65
201449.6 11.26
200999.6 10.14
201149.6 11.37
200699.6 8.55
200849.6 10.40
200399.6 7.32
200549.6 7.15
200099.6 12.96
200249.6 10.90
199799.6 8.79
199949.6 11.68
199499.6 8.86
199649.6 6.66
199199.6 10.60
199349.6 7.32
198899.6 6.37
199049.6 6.68
198599.6 10.77
198749.6 9.91
198299.6 10.47
198449.6 10.81
197999.6 9.12
198149.6 11.41
197699.6 8.42
197849.6 10.05
197399.6 7.72
197549.6 10.20
197099.6 9.73
197249.6 11.79
196799.6 10.92
196949.6 10.37
196499.6 10.77
196649.6 7.37
196199.6 6.28
196349.6 7.66
195899.6 11.91
196049.6 6.48
195599.6 8.80
195749.6 6.72
195299.6 9.98
195449.6 7.94
194999.6 8.25
195149.6 6.90
194699.6 9.29
194849.6 9.24
194399.6 9.93
194549.6 11.36
194099.6 7.70
194249.6 9.19
193799.6 7.66
193949.6 8.48
193499.6 7.41
193649.6 10.51
193199.6 10.99
193349.6 10.54
192899.6 6.98
193049.6 10.41
192599.6 7.95
192749.6 8.10
192299.6 10.20
192449.6 5.86
191999.6 6.32
192149.6 9.97
191699.6 11.10
191849.6 10.42
191399.6 10.74
191549.6 9.11
191099.6 9.74
191249.6 8.23
190799.6 6.73
190949.6 8.84
190499.6 11.77
190649.6 7.73
190199.6 5.78
190349.6 9.12
189899.6 5.84
190049.6 9.64
189599.6 10.05
189749.6 9.16
189299.6 11.25
189449.6 11.40
188999.6 9.56
189149.6 8.42
188699.6 10.62
188849.6 7.05
188399.6 8.02
188549.6 8.94
188099.6 5.78
188249.6 7.79
187799.6 10.49
187949.6 10.57
187499.6 11.26
187649.6 8.59
187199.6 6.37
187349.6 11.47
186899.6 6.82
187049.6 9.40
186599.6 6.42
186749.6 7.21
186299.6 7.76
186449.6 7.01
185999.6 11.04
186149.6 10.53
185699.6 11.54
185849.6 7.52
185399.6 7.44
185549.6 8.61
185099.6 9.43
185249.6 9.15
184799.6 6.91
184949.6 11.58
184499.6 9.15
184649.6 6.99
184199.6 11.29
184349.6 6.97
183899.6 9.10
184049.6 7.30
183599.6 8.83
183749.6 5.74
183299.6 9.85
183449.6 8.88
182999.6 10.57
183149.6 6.11
182699.6 5.94
182849.6 6.01
182399.6 9.58
182549.6 8.51
182099.6 5.89
182249.6 10.65
181799.6 6.39
181949.6 7.21
181499.6 8.38
181649.6 6.65
181199.6 7.79
181349.6 6.47
180899.6 10.62
181049.6 9.74
180599.6 6.51
180749.6 10.01
180299.6 9.73
180449.6 8.14
179999.6 5.88
180149.6 7.63
179699.6 5.73
179849.6 9.63
179399.6 10.06
179549.6 8.97
179099.6 9.73
179249.6 6.73
178799.6 8.00
178949.6 10.16
178499.6 7.76
178649.6 10.85
178199.6 8.25
178349.6 7.86
177899.6 8.26
178049.6 10.51
177599.6 6.03
177749.6 10.01
177299.6 9.08
177449.6 6.60
176999.6 6.08
177149.6 6.87
176699.6 6.97
176849.6 7.85
176399.6 7.95
176549.6 10.53
176099.6 6.97
176249.6 5.76
175799.6 6.52
175949.6 9.38
175499.6 7.73
175649.6 5.91
175199.6 5.63
175349.6 9.37
174899.6 6.65
175049.6 8.00
174599.6 5.50
174749.6 8.45
174299.6 9.80
174449.6 6.76
173999.6 8.84
174149.6 9.23
173699.6 10.00
173849.6 6.86
173399.6 11.44
173549.6 6.10
173099.6 9.50
173249.6 10.02
172799.6 10.51
172949.6 6.12
172499.6 10.89
172649.6 6.87
172199.6 8.73
172349.6 10.07
171899.6 10.40
172049.6 5.44
171599.6 5.63
171749.6 7.46
171299.6 8.79
171449.6 6.80
170999.6 6.96
171149.6 8.87
170699.6 11.18
170849.6 7.72
170399.6 5.59
170549.6 9.64
170099.6 10.07
170249.6 9.33
169799.6 9.39
169949.6 9.25
169499.6 5.45
169649.6 9.20
169199.6 8.76
169349.6 10.92
168899.6 8.18
169049.6 6.78
168599.6 9.29
168749.6 6.03
168299.6 10.56
168449.6 8.75
167999.6 10.34
168149.6 6.49
167699.6 6.06
167849.6 5.74
167399.6 8.37
167549.6 9.14
167099.6 6.64
167249.6 10.26
166799.6 7.23
166949.6 6.75
166499.6 5.74
166649.6 8.90
166199.6 8.20
166349.6 5.72
165899.6 5.81
166049.6 6.37
165599.6 9.61
165749.6 6.19
165299.6 7.34
165449.6 6.36
164999.6 5.88
165149.6 9.04
164699.6 6.55
164849.6 9.20
164399.6 10.04
164549.6 8.84
164099.6 9.45
164249.6 6.43
163799.6 9.16
163949.6 9.59
163499.6 10.94
163649.6 7.72
163199.6 9.29
163349.6 6.67
162899.6 8.92
163049.6 9.01
162599.6 5.48
162749.6 11.09
162299.6 5.89
162449.6 8.74
161999.6 7.51
162149.6 5.38
161699.6 8.39
161849.6 8.50
161399.6 5.57
161549.6 6.20
161099.6 7.09
161249.6 10.83
160799.6 8.89
160949.6 7.82
160499.6 10.92
160649.6 8.86
160199.6 5.80
160349.6 8.81
159899.6 9.13
160049.6 5.20
159599.6 7.89
159749.6 6.40
159299.6 9.64
159449.6 7.29
158999.6 9.32
159149.6 8.38
158699.6 10.71
158849.6 8.56
158399.6 9.96
158549.6 5.82
158099.6 8.31
158249.6 10.53
157799.6 5.23
157949.6 11.07
157499.6 6.16
157649.6 6.17
157199.6 6.58
157349.6 6.11
156899.6 10.98
157049.6 10.96
156599.6 6.56
156749.6 5.92
156299.6 10.11
156449.6 5.81
155999.6 10.46
156149.6 5.39
155699.6 5.18
155849.6 7.59
155399.6 6.92
155549.6 8.10
155099.6 10.52
155249.6 8.11
154799.6 10.72
154949.6 7.60
154499.6 10.45
154649.6 5.85
154199.6 8.70
154349.6 5.12
153899.6 9.73
154049.6 5.08
153599.6 6.99
153749.6 7.52
153299.6 10.48
153449.6 9.40
152999.6 6.48
153149.6 10.46
152699.6 10.14
152849.6 7.52
152399.6 5.77
152549.6 5.16
152099.6 8.08
152249.6 9.62
151799.6 6.92
151949.6 6.35
151499.6 6.56
151649.6 10.14
151199.6 10.30
151349.6 7.62
150899.6 5.28
151049.6 7.46
150599.6 6.48
150749.6 9.48
150299.6 5.07
150449.6 7.82
149999.6 10.91
150149.6 10.84
149699.6 9.36
149849.6 6.29
149399.6 10.67
149549.6 8.07
149099.6 7.60
149249.6 7.71
148799.6 6.89
148949.6 6.43
148499.6 7.10
148649.6 5.95
148199.6 6.82
148349.6 7.66
147899.6 8.48
148049.6 5.38
147599.6 8.38
147749.6 6.39
147299.6 9.09
147449.6 5.83
146999.6 10.94
147149.6 8.77
146699.6 7.30
146849.6 9.46
146399.6 5.08
146549.6 6.55
146099.6 9.86
146249.6 7.48
145799.6 8.77
145949.6 8.52
145499.6 7.15
145649.6 6.77
145199.6 7.57
145349.6 7.14
144899.6 8.55
145049.6 5.05
144599.6 7.61
144749.6 4.90
144299.6 8.97
144449.6 6.89
143999.6 8.24
144149.6 5.05
143699.6 9.47
143849.6 5.86
143399.6 10.70
143549.6 5.23
143099.6 9.71
143249.6 5.88
142799.6 6.11
142949.6 6.45
142499.6 8.10
142649.6 5.57
142199.6 9.32
142349.6 5.35
141899.6 6.66
142049.6 4.89
141599.6 9.96
141749.6 5.31
141299.6 7.94
141449.6 5.12
140999.6 5.42
141149.6 6.91
140699.6 6.00
140849.6 4.89
140399.6 8.49
140549.6 10.62
140099.6 9.97
140249.6 5.90
139799.6 9.91
139949.6 8.41
139499.6 9.18
139649.6 7.72
139199.6 5.33
139349.6 10.72
138899.6 9.15
139049.6 9.78
138599.6 10.67
138749.6 7.89
138299.6 4.80
138449.6 5.86
137999.6 8.29
138149.6 5.02
137699.6 8.54
137849.6 7.45
137399.6 5.19
137549.6 7.13
137099.6 9.78
137249.6 6.89
136799.6 6.20
136949.6 10.21
136499.6 9.18
136649.6 9.64
136199.6 4.96
136349.6 7.37
135899.6 8.39
136049.6 8.28
135599.6 10.61
135749.6 8.75
135299.6 8.16
135449.6 6.61
134999.5 9.41
135149.6 8.92
134699.5 5.29
134849.5 5.36
134399.5 5.52
134549.5 5.38
134099.5 4.90
134249.5 7.42
133799.5 5.09
133949.5 10.18
133499.5 7.34
133649.5 6.61
133199.5 4.91
133349.5 5.75
132899.5 7.87
133049.5 10.46
132599.5 8.46
132749.5 10.00
132299.5 10.60
132449.5 6.17
131999.5 10.12
132149.5 6.60
131699.5 7.09
131849.5 5.23
131399.5 9.80
131549.5 5.84
131099.5 6.04
131249.5 7.55
130799.5 7.00
130949.5 9.73
130499.5 5.28
130649.5 6.99
130199.5 4.93
130349.5 7.64
129899.5 9.84
130049.5 6.82
129599.5 9.81
129749.5 6.85
129299.5 5.51
129449.5 7.06
128999.5 6.57
129149.5 8.95
128699.5 9.73
128849.5 9.09
128399.5 5.74
128549.5 5.15
128099.5 5.14
128249.5 6.96
127799.5 8.02
127949.5 8.95
127499.5 10.16
127649.5 8.17
127199.5 5.62
127349.5 5.27
126899.5 9.27
127049.5 5.69
126599.5 7.19
126749.5 6.84
126299.5 7.21
126449.5 5.11
125999.5 5.31
126149.5 5.58
125699.5 10.29
125849.5 7.86
125399.5 5.11
125549.5 5.73
125099.5 8.13
125249.5 10.43
124799.5 8.00
124949.5 8.66
124499.5 5.93
124649.5 9.15
124199.5 7.61
124349.5 6.53
123899.5 9.49
124049.5 8.84
123599.5 8.18
123749.5 8.48
123299.5 7.43
123449.5 4.84
122999.5 9.70
123149.5 7.25
122699.5 9.74
122849.5 8.58
122399.5 7.35
122549.5 9.31
122099.5 5.84
122249.5 9.62
121799.5 5.33
121949.5 7.34
121499.5 6.17
121649.5 8.05
121199.5 6.05
121349.5 8.75
120899.5 9.45
121049.5 7.45
120599.5 7.38
120749.5 4.94
120299.5 5.82
120449.5 5.55
119999.5 9.99
120149.5 7.87
119699.5 7.42
119849.5 9.24
119399.5 10.39
119549.5 7.93
119099.5 7.60
119249.5 6.71
118799.5 4.49
118949.5 4.42
118499.5 8.08
118649.5 8.59
118199.5 9.62
118349.5 8.71
117899.5 6.28
118049.5 5.21
117599.5 5.55
117749.5 6.18
117299.5 8.73
117449.5 7.94
116999.5 9.92
117149.5 8.41
116699.5 4.51
116849.5 9.92
116399.5 7.08
116549.5 7.71
116099.5 7.94
116249.5 8.72
115799.5 4.99
115949.5 7.33
115499.5 10.09
115649.5 6.14
115199.5 9.37
115349.5 8.95
114899.5 10.04
115049.5 6.87
114599.5 4.55
114749.5 9.33
114299.5 4.59
114449.5 4.51
113999.5 6.86
114149.5 5.47
113699.5 9.61
113849.5 7.29
113399.5 10.13
113549.5 6.70
113099.5 8.32
113249.5 9.83
112799.5 9.64
112949.5 8.68
112499.5 8.51
112649.5 8.39
112199.5 7.60
112349.5 6.42
111899.5 5.80
112049.5 9.24
111599.5 8.92
111749.5 9.41
111299.5 8.36
111449.5 8.54
110999.5 6.75
111149.5 4.44
110699.5 8.43
110849.5 9.37
110399.5 7.39
110549.5 5.17
110099.5 5.85
110249.5 4.81
109799.5 9.97
109949.5 10.17
109499.5 9.94
109649.5 5.14
109199.5 4.65
109349.5 7.42
108899.5 5.55
109049.5 4.34
108599.5 7.03
108749.5 9.88
108299.5 7.55
108449.5 4.77
107999.5 7.09
108149.5 8.95
107699.5 7.68
107849.5 5.26
107399.5 6.17
107549.5 4.74
107099.5 6.76
107249.5 5.56
106799.5 7.26
106949.5 5.96
106499.5 10.07
106649.5 5.68
106199.5 7.96
106349.5 8.94
105899.5 5.97
106049.5 6.35
105599.5 5.01
105749.5 5.39
105299.5 5.37
105449.5 7.45
104999.5 5.08
105149.5 5.59
104699.5 9.56
104849.5 8.77
104399.5 9.25
104549.5 10.10
104099.5 7.80
104249.5 8.06
103799.5 4.53
103949.5 6.13
103499.5 9.73
103649.5 8.60
103199.5 8.83
103349.5 7.71
102899.5 5.66
103049.5 7.02
102599.5 8.10
102749.5 8.56
102299.5 8.87
102449.5 9.51
101999.5 5.32
102149.5 4.86
101699.5 6.50
101849.5 5.82
101399.5 8.34
101549.5 8.32
101099.5 5.36
101249.5 4.63
100799.5 7.03
100949.5 6.21
100499.5 5.20
100649.5 7.13
100199.5 9.40
100349.5 7.50
99899.5 4.83
100049.5 8.15
99599.5 4.68
99749.5 5.38
99299.5 5.37
99449.5 6.95
98999.5 8.09
99149.5 8.89
98699.5 8.22
98849.5 7.64
98399.5 8.47
98549.5 4.66
98099.5 6.77
98249.5 5.55
97799.5 7.09
97949.5 9.81
97499.5 7.89
97649.5 5.01
97199.5 6.41
97349.5 9.53
96899.5 6.05
97049.5 6.58
96599.5 9.79
96749.5 6.97
96299.5 9.33
96449.5 6.88
95999.5 4.54
96149.5 8.47
95699.5 9.16
95849.5 4.69
95399.5 8.16
95549.5 5.87
95099.5 5.56
95249.5 5.34
94799.5 5.18
94949.5 7.14
94499.5 8.99
94649.5 9.14
94199.5 6.54
94349.5 4.31
93899.5 6.88
94049.5 4.02
93599.5 8.16
93749.5 4.17
93299.5 8.13
93449.5 6.94
92999.5 4.49
93149.5 9.43
92699.5 7.12
92849.5 9.79
92399.5 7.43
92549.5 4.89
92099.5 9.70
92249.5 8.56
91799.5 8.64
91949.5 8.33
91499.5 4.40
91649.5 6.26
91199.5 5.72
91349.5 6.61
90899.5 4.16
91049.5 5.04
90599.5 6.24
90749.5 7.78
90299.5 4.22
90449.5 6.53
89999.5 6.10
90149.5 6.22
89699.5 6.93
89849.5 8.82
89399.5 9.12
89549.5 8.43
89099.5 8.81
89249.5 7.10
88799.5 4.32
88949.5 7.40
88499.5 4.11
88649.5 6.58
88199.5 5.73
88349.5 4.14
87899.5 7.61
88049.5 6.13
87599.5 7.89
87749.5 8.77
87299.5 8.87
87449.5 4.35
86999.5 6.32
87149.5 9.11
86699.5 8.32
86849.5 8.19
86399.5 3.95
86549.5 6.97
86099.5 5.35
86249.5 6.23
85799.5 5.19
85949.5 8.88
85499.5 6.01
85649.5 7.52
85199.5 8.85
85349.5 7.81
84899.5 9.13
85049.5 7.11
84599.5 4.32
84749.5 8.37
84299.5 8.77
84449.5 7.53
83999.5 5.80
84149.5 5.19
83699.5 6.23
83849.5 8.56
83399.5 6.94
83549.5 4.13
83099.5 6.25
83249.5 7.06
82799.5 9.39
82949.5 7.27
82499.5 8.06
82649.5 5.33
82199.5 8.23
82349.5 7.47
81899.5 5.54
82049.5 4.54
81599.5 5.98
81749.5 4.68
81299.5 6.04
81449.5 5.94
80999.5 4.98
81149.5 9.35
80699.5 4.91
80849.5 7.96
80399.5 9.56
80549.5 7.33
80099.5 5.68
80249.5 4.66
79799.5 7.80
79949.5 6.47
79499.5 4.29
79649.5 4.93
79199.5 5.86
79349.5 4.80
78899.5 6.91
79049.5 6.96
78599.5 6.43
78749.5 8.25
78299.5 4.14
78449.5 6.49
77999.5 5.75
78149.5 5.15
77699.5 6.57
77849.5 4.29
77399.5 7.14
77549.5 8.92
77099.5 8.19
77249.5 4.07
76799.5 9.29
76949.5 7.37
76499.5 4.05
76649.5 4.03
76199.5 8.98
76349.5 8.45
75899.5 7.93
76049.5 5.74
75599.5 7.21
75749.5 3.68
75299.5 7.48
75449.5 9.59
74999.5 6.95
75149.5 5.85
74699.5 8.06
74849.5 7.26
74399.5 4.06
74549.5 6.60
74099.5 8.10
74249.5 6.19
73799.5 9.35
73949.5 5.06
73499.5 7.04
73649.5 9.44
73199.5 4.25
73349.5 6.41
72899.5 7.99
73049.5 9.53
72599.5 4.29
72749.5 5.49
72299.5 5.27
72449.5 6.61
71999.5 9.06
72149.5 7.83
71699.5 8.26
71849.5 7.34
71399.5 5.14
71549.5 8.63
71099.5 3.81
71249.5 7.14
70799.5 8.42
70949.5 4.41
70499.5 3.80
70649.5 5.75
70199.5 3.85
70349.5 6.94
69899.5 9.02
70049.5 7.64
69599.5 6.89
69749.5 4.54
69299.5 7.68
69449.5 6.50
68999.5 4.16
69149.5 9.40
68699.5 8.97
68849.5 7.92
68399.5 4.83
68549.5 4.96
68099.5 6.67
68249.5 6.15
67799.5 9.02
67949.5 7.56
67499.5 4.15
67649.5 6.51
67199.5 8.59
67349.5 5.79
66899.5 8.71
67049.5 8.64
66599.5 6.08
66749.5 7.51
66299.5 5.60
66449.5 5.29
65999.5 4.69
66149.5 8.06
65699.5 3.81
65849.5 6.02
65399.5 3.90
65549.5 6.70
65099.5 7.72
65249.5 8.94
64799.5 6.97
64949.5 4.61
64499.5 4.83
64649.5 9.38
64199.5 4.78
64349.5 8.60
63899.5 4.20
64049.5 6.70
63599.5 9.39
63749.5 6.56
63299.5 6.19
63449.5 7.84
62999.5 5.40
63149.5 4.44
62699.5 6.88
62849.5 3.78
62399.5 5.17
62549.5 8.43
62099.5 8.92
62249.5 5.64
61799.5 4.03
61949.5 6.99
61499.5 7.83
61649.5 8.83
61199.5 5.37
61349.5 5.45
60899.5 8.17
61049.5 7.76
60599.5 7.86
60749.5 4.56
60299.5 6.74
60449.5 4.16
59999.5 6.06
60149.5 7.75
59699.5 6.87
59849.5 4.43
59399.5 5.78
59549.5 8.29
59099.5 6.41
59249.5 4.25
58799.5 3.79
58949.5 7.24
58499.5 6.12
58649.5 6.56
58199.5 8.96
58349.5 6.52
57899.5 6.59
58049.5 4.30
57599.5 3.46
57749.5 4.68
57299.5 6.10
57449.5 7.95
56999.5 7.04
57149.5 4.07
56699.5 6.77
56849.5 3.88
56399.5 6.35
56549.5 4.66
56099.5 4.45
56249.5 5.21
55799.5 8.49
55949.5 4.01
55499.5 4.10
55649.5 6.80
55199.5 8.59
55349.5 5.41
54899.5 5.33
55049.5 4.06
54599.5 6.60
54749.5 4.29
54299.5 4.53
54449.5 7.68
53999.5 8.84
54149.5 6.84
53699.5 7.36
53849.5 3.94
53399.5 3.73
53549.5 6.92
53099.5 7.97
53249.5 9.07
52799.5 4.56
52949.5 3.51
52499.5 4.20
52649.5 3.87
52199.5 4.39
52349.5 6.27
51899.5 4.21
52049.5 5.42
51599.5 3.37
51749.5 3.32
51299.5 7.29
51449.5 3.55
50999.5 7.25
51149.5 3.65
50699.5 5.76
50849.5 8.23
50399.5 6.07
50549.5 5.25
50099.5 8.11
50249.5 5.76
49799.5 3.33
49949.5 7.33
49499.5 8.85
49649.5 3.74
49199.5 7.27
49349.5 8.68
48899.5 6.33
49049.5 4.35
48599.5 6.46
48749.5 4.43
48299.5 3.71
48449.5 7.24
47999.5 5.64
48149.5 8.94
47699.5 6.63
47849.5 7.01
47399.5 6.05
47549.5 4.61
47099.5 3.47
47249.5 6.08
46799.5 8.68
46949.5 5.59
46499.5 4.58
46649.5 5.43
46199.5 3.17
46349.5 7.73
45899.5 8.52
46049.5 4.56
45599.5 6.80
45749.5 7.64
45299.5 8.08
45449.5 3.64
44999.5 4.10
45149.5 6.56
44699.5 7.47
44849.5 8.85
44399.5 5.93
44549.5 3.88
44099.5 5.73
44249.5 7.32
43799.5 4.06
43949.5 6.84
43499.5 8.38
43649.5 4.92
43199.5 6.20
43349.5 5.22
42899.5 7.98
43049.5 6.48
42599.5 6.40
42749.5 4.26
42299.5 5.89
42449.5 4.64
41999.5 7.86
42149.5 4.92
41699.5 3.51
41849.5 6.35
41399.5 8.86
41549.5 7.80
41099.5 8.08
41249.5 3.95
40799.5 4.44
40949.5 5.28
40499.5 4.62
40649.5 4.80
40199.5 7.13
40349.5 4.64
39899.5 5.38
40049.5 5.43
39599.5 7.63
39749.5 8.39
39299.5 6.36
39449.5 5.72
38999.5 8.81
39149.5 8.39
38699.5 7.56
38849.5 7.76
38399.5 6.65
38549.5 6.01
38099.5 7.99
38249.5 6.37
37799.5 5.18
37949.5 7.54
37499.5 3.10
37649.5 7.13
37199.5 5.96
37349.5 7.41
36899.5 6.79
37049.5 4.89
36599.5 3.32
36749.5 3.21
36299.5 4.99
36449.5 7.65
35999.5 6.16
36149.5 7.59
35699.5 6.77
35849.5 8.47
35399.5 4.67
35549.5 7.27
35099.5 8.43
35249.5 7.28
34799.5 8.22
34949.5 6.67
34499.5 3.28
34649.5 6.43
34199.5 6.04
34349.5 8.26
33899.5 3.08
34049.5 6.04
33599.5 4.76
33749.5 6.81
33299.5 4.80
33449.5 4.42
32999.5 5.29
33149.5 6.63
32699.5 5.58
32849.5 3.87
32399.5 6.32
32549.5 7.46
32099.5 3.95
32249.5 4.77
31799.5 5.97
31949.5 3.58
31499.5 7.68
31649.5 8.21
31199.5 4.60
31349.5 5.86
30899.5 8.43
31049.5 6.08
30599.5 3.79
30749.5 5.25
30299.5 7.05
30449.5 2.80
29999.5 4.50
30149.5 5.25
29699.5 3.77
29849.5 3.09
29399.5 6.43
29549.5 4.99
29099.5 6.73
29249.5 3.61
28799.5 6.31
28949.5 7.01
28499.5 5.48
28649.5 6.95
28199.5 8.47
28349.5 5.56
27899.5 7.20
28049.5 3.63
27599.5 6.30
27749.5 4.58
27299.5 3.80
27449.5 3.37
26999.5 4.66
27149.5 3.58
26699.5 7.98
26849.5 5.51
26399.5 5.10
26549.5 7.64
26099.5 5.39
26249.5 7.64
25799.5 6.58
25949.5 8.32
25499.5 6.11
25649.5 3.82
25199.5 4.16
25349.5 7.85
24899.5 5.97
25049.5 4.69
24599.5 3.66
24749.5 5.67
24299.5 7.46
24449.5 5.15
23999.5 4.55
24149.5 6.97
23699.5 2.55
23849.5 6.76
23399.5 3.20
23549.5 3.13
23099.5 3.32
23249.5 4.53
22799.5 6.82
22949.5 6.83
22499.5 4.00
22649.5 6.06
22199.5 5.77
22349.5 7.29
21899.5 8.20
22049.5 2.97
21599.5 3.26
21749.5 3.96
21299.5 6.22
21449.5 6.47
20999.5 4.93
21149.5 8.61
20699.5 8.18
20849.5 6.56
20399.5 4.71
20549.5 7.84
20099.5 5.19
20249.5 3.83
19799.5 4.21
19949.5 5.19
19499.5 4.66
19649.5 5.36
19199.5 7.20
19349.5 6.96
18899.5 6.22
19049.5 8.13
18599.5 6.87
18749.5 5.21
18299.5 4.33
18449.5 5.22
17999.5 6.57
18149.5 5.21
17699.4 4.67
17849.4 4.10
17399.4 8.42
17549.4 5.86
17099.4 3.53
17249.4 5.52
16799.4 7.22
16949.4 8.25
16499.4 7.68
16649.4 7.51
16199.4 6.84
16349.4 4.62
15899.4 3.45
16049.4 4.74
15599.4 6.82
15749.4 8.37
15299.4 8.23
15449.4 7.30
14999.4 5.99
15149.4 3.35
14699.4 4.68
14849.4 2.87
14399.4 5.53
14549.4 6.04
14099.4 7.99
14249.4 8.27
13799.4 5.58
13949.4 7.00
13499.4 7.58
13649.4 5.78
13199.4 7.07
13349.4 6.93
12899.4 2.78
13049.4 7.73
12599.4 7.84
12749.4 2.54
12299.4 7.45
12449.4 6.35
11999.4 6.08
12149.4 7.47
11699.4 4.09
11849.4 3.92
11399.4 6.71
11549.4 6.45
11099.4 5.35
11249.4 7.54
10799.4 7.99
10949.4 6.23
10499.4 6.02
10649.4 4.43
10199.4 5.74
10349.4 6.85
9899.4 4.52
10049.4 5.77
9599.4 5.30
9749.4 3.34
9299.4 7.72
9449.4 5.40
8999.4 7.63
9149.4 4.22
8699.4 3.80
8849.4 5.52
8399.4 4.46
8549.4 7.65
8099.4 4.71
8249.4 2.62
7799.4 4.57
7949.4 3.78
7499.4 5.21
7649.4 5.25
7199.4 4.31
7349.4 2.18
6899.4 3.59
7049.4 7.08
6599.4 2.79
6749.4 5.13
6299.4 7.44
6449.4 5.56
5999.4 3.82
6149.4 3.39
5699.4 3.33
5849.4 4.88
5399.4 5.07
5549.4 2.20
5099.4 6.37
5249.4 1.96
4799.4 5.89
4949.4 4.04
4499.4 7.57
4649.4 2.80
4199.4 2.02
4349.4 6.22
3899.4 6.82
4049.4 6.60
3599.4 7.45
3749.4 5.24
3299.4 3.13
3449.4 6.75
2999.4 7.31
3149.4 4.33
2699.4 3.54
2849.4 4.53
2399.4 4.03
2549.4 4.78
2099.4 2.04
2249.4 7.35
1799.4 2.60
1949.4 6.65
1499.4 2.70
1649.4 5.80
1199.4 4.21
1349.4 7.26
899.4 5.16
1049.4 3.75
599.4 1.78
749.4 1.56
299.4 3.09
449.4 6.11

"Zone bandwidth for /dev/da1
1.1 128.63
2002.1 74.02
4003.1 74.05
6004.2 74.05
8005.2 72.80
10006.3 72.84
12007.3 74.04
14008.4 74.02
16009.4 74.02
18010.5 74.03
20011.5 72.83
22012.6 72.84
24013.6 74.05
26014.7 73.46
28015.7 73.46
30016.8 73.46
32017.8 73.48
34018.9 74.61
36019.9 75.93
38021.0 73.47
40022.0 73.46
42023.0 73.45
44024.1 75.93
46025.1 75.91
48026.2 72.25
50027.2 72.24
52028.3 72.28
54029.3 72.34
56030.4 74.05
58031.4 72.50
60032.5 72.85
62033.5 70.87
64034.6 72.50
66035.6 72.82
68036.7 72.46
70037.7 74.01
72038.8 69.74
74039.8 71.31
76040.9 69.71
78041.9 72.49
80043.0 74.06
82044.0 70.86
84045.0 74.04
86046.1 69.77
88047.1 69.75
90048.2 69.74
92049.2 72.84
94050.3 69.73
96051.3 70.86
98052.4 69.75
100053.4 74.05
102054.5 70.85
104055.5 70.86
106056.6 69.81
108057.6 69.76
110058.7 68.43
112059.7 70.86
114060.8 68.90
116061.8 65.44
118062.9 69.72
120063.9 69.51
122065.0 70.85
124066.0 68.92
126067.0 68.90
128068.1 70.85
130069.1 67.55
132070.2 69.14
134071.2 67.49
136072.3 68.08
138073.3 68.58
140074.4 69.15
142075.4 68.58
144076.5 68.09
146077.5 68.12
148078.6 69.71
150079.6 66.39
152080.7 66.29
154081.7 67.38
156082.8 69.63
158083.8 69.59
160084.9 67.40
162085.9 66.40
164086.9 65.19
166088.0 69.64
168089.0 68.53
170090.1 65.17
172091.1 68.10
174092.2 66.89
176093.2 65.12
178094.3 64.16
180095.3 65.84
182096.4 68.13
184097.4 65.12
186098.5 62.65
188099.5 66.85
190100.6 65.12
192101.6 65.12
194102.7 65.83
196103.7 63.53
198104.8 62.65
200105.8 62.65
202106.9 63.57
204107.9 65.18
206108.9 63.67
208110.0 63.72
210111.0 61.51
212112.1 61.49
214113.1 64.41
216114.2 61.05
218115.2 59.99
220116.3 60.80
222117.3 61.07
224118.4 61.06
226119.4 64.42
228120.5 61.79
230121.5 58.63
232122.6 61.78
234123.6 60.00
236124.7 58.64
238125.7 58.62
240126.8 59.98
242127.8 57.21
244128.9 57.78
246129.9 57.79
248130.9 59.99
250132.0 57.77
252133.0 59.98
254134.1 57.77
256135.1 59.99
258136.2 57.77
260137.2 56.51
262138.3 56.49
264139.3 54.28
266140.4 53.61
268141.4 55.30
270142.5 56.50
272143.5 57.78
274144.6 54.27
276145.6 54.28
278146.7 54.27
280147.7 54.25
282148.8 54.27
284149.8 54.29
286150.9 54.27
288151.9 53.61
290152.9 54.27
292154.0 50.55
294155.0 50.58
296156.1 51.97
298157.1 49.96

UDP latency using localhost: 26.3394 microseconds
TCP latency using localhost: 26.8483 microseconds
RPC/udp latency using localhost: 41.5717 microseconds
RPC/tcp latency using localhost: 38.5316 microseconds
connect: Address already in use

Socket bandwidth using localhost
0.000001 2.22 MB/sec
0.000064 26.27 MB/sec
0.000128 43.43 MB/sec
0.000256 147.44 MB/sec
0.000512 268.46 MB/sec
0.001024 417.13 MB/sec
0.001437 505.58 MB/sec
10.000000 985.71 MB/sec

Avg xfer: 3.2KB, 41.8KB in 205 millisecs, 203.97 KB/sec
AF_UNIX sock stream bandwidth: 447.19 MB/sec
Pipe bandwidth: 2521.30 MB/sec

"read bandwidth
0.000512 503.01
0.001024 938.84
0.002048 1657.96
0.004096 2707.54
0.008192 3947.70
0.016384 4477.45
0.032768 4501.69
0.065536 4709.17
0.131072 4706.01
0.262144 4564.81
0.524288 4491.49
1.05 4473.92
2.10 4460.44
4.19 4447.83
8.39 3769.03
16.78 3115.55
33.55 3152.43
67.11 3149.32
134.22 3128.84
268.44 3344.24
536.87 3471.70
1073.74 3493.17
2147.48 3471.77

"read open2close bandwidth
0.000512 79.05
0.001024 156.28
0.002048 303.16
0.004096 585.68
0.008192 1068.59
0.016384 1702.74
0.032768 2484.24
0.065536 3195.11
0.131072 3725.68
0.262144 3984.97
0.524288 4194.30
1.05 4320.54
2.10 4375.90
4.19 4402.70
8.39 3706.85
16.78 3092.57
33.55 3143.27
67.11 3142.24
134.22 3125.85
268.44 3341.28
536.87 3470.62
1073.74 3493.18
2147.48 3470.70


"Mmap read bandwidth
0.000512 13443.44
0.001024 10958.59
0.002048 9901.59
0.004096 9448.58
0.008192 9234.84
0.016384 9133.68
0.032768 9052.20
0.065536 9043.16
0.131072 9028.77
0.262144 8853.62
0.524288 8607.43
1.05 8585.50
2.10 8571.97
4.19 8708.45
8.39 6930.44
16.78 5326.10
33.55 5423.38
67.11 5426.89
134.22 5389.84
268.44 5858.99
536.87 6170.86
1073.74 6339.77
2147.48 6426.99

"Mmap read open2close bandwidth
0.000512 38.03
0.001024 73.89
0.002048 151.92
0.004096 296.05
0.008192 421.82
0.016384 739.90
0.032768 985.19
0.065536 1107.95
0.131072 1213.19
0.262144 1285.95
0.524288 871.63
1.05 826.56
2.10 825.54
4.19 864.54
8.39 1033.84
16.78 915.29
33.55 948.24
67.11 899.12
134.22 1022.73
268.44 869.56
536.87 792.45
1073.74 758.61
2147.48 742.70


"libc bcopy unaligned
0.000512 11560.81
0.001024 17785.01
0.002048 23832.57
0.004096 28715.99
0.008192 31999.16
0.016384 32690.44
0.032768 14828.35
0.065536 15174.28
0.131072 14873.42
0.262144 13521.01
0.524288 10802.25
1.05 10802.25
2.10 10816.04
4.19 7469.08
8.39 4262.50
16.78 4257.64
33.55 4536.22
67.11 4309.31
134.22 4213.26
268.44 4369.07
536.87 3407.08
1073.74 3070.16

"libc bcopy aligned
0.000512 11559.33
0.001024 17785.67
0.002048 23835.98
0.004096 28715.99
0.008192 31999.16
0.016384 32507.89
0.032768 14934.38
0.065536 14961.83
0.131072 14854.01
0.262144 11116.20
0.524288 9654.70
1.05 9607.86
2.10 9644.74
4.19 7570.95
8.39 3378.87
16.78 3748.68
33.55 2927.45
67.11 3324.85
134.22 4139.07
268.44 4375.40
536.87 4370.35
1073.74 3965.07

Memory bzero bandwidth
0.000512 9880.71
0.001024 15726.43
0.002048 21890.86
0.004096 27285.78
0.008192 31086.98
0.016384 33411.21
0.032768 34445.20
0.065536 17359.89
0.131072 16226.28
0.262144 15590.39
0.524288 15235.88
1.05 15042.22
2.10 14966.29
4.19 14926.35
8.39 12746.22
16.78 7458.75
33.55 7673.09
67.11 8296.31
134.22 7806.08
268.44 7992.00
536.87 8107.63
1073.74 8163.10
2147.48 7674.85

"unrolled bcopy unaligned
0.000512 8437.74
0.001024 8774.05
0.002048 8875.54
0.004096 8966.05
0.008192 8992.86
0.016384 8767.56
0.032768 6629.05
0.065536 6622.94
0.131072 6604.97
0.262144 5888.19
0.524288 5863.25
1.05 5870.66
2.10 5882.61
4.19 5344.03
8.39 3091.62
16.78 3458.51
33.55 3438.66
67.11 2988.99
134.22 3160.89
268.44 3298.30
536.87 3317.85
1073.74 3017.42

"unrolled partial bcopy unaligned
0.000512 28196.55
0.001024 33268.52
0.002048 33750.97
0.004096 35365.94
0.008192 35503.64
0.016384 31618.56
0.032768 13855.05
0.065536 13955.62
0.131072 13824.75
0.262144 9268.44
0.524288 9302.96
1.05 9308.01
2.10 9291.42
4.19 7479.14
8.39 4116.10
16.78 3866.61
33.55 3833.04
67.11 3920.83
134.22 3878.23
268.44 3930.41
536.87 4042.25
1073.74 3631.63

Memory read bandwidth
0.000512 5687.67
0.001024 5474.73
0.002048 5729.93
0.004096 5729.65
0.008192 5728.73
0.016384 5726.22
0.032768 5721.59
0.065536 5710.70
0.131072 5715.35
0.262144 5710.16
0.524288 5543.55
1.05 5544.99
2.10 5542.16
4.19 5556.29
8.39 5285.83
16.78 4870.73
33.55 4680.49
67.11 4643.89
134.22 4818.62
268.44 4755.78
536.87 4780.39
1073.74 4814.14
2147.48 4605.21

Memory partial read bandwidth
0.000512 32111.85
0.001024 33996.99
0.002048 34689.30
0.004096 35124.54
0.008192 35409.67
0.016384 35560.25
0.032768 35266.20
0.065536 24013.64
0.131072 24038.25
0.262144 23969.90
0.524288 18032.84
1.05 17974.51
2.10 17946.75
4.19 18101.65
8.39 14699.66
16.78 9949.42
33.55 8172.05
67.11 9363.59
134.22 8992.21
268.44 9158.49
536.87 9348.10
1073.74 9670.04
2147.48 8248.42

Memory write bandwidth
0.000512 9031.38
0.001024 9031.63
0.002048 9031.63
0.004096 9031.98
0.008192 9026.70
0.016384 9031.63
0.032768 9018.99
0.065536 8492.76
0.131072 8496.95
0.262144 8488.75
0.524288 8041.23
1.05 8038.65
2.10 8035.07
4.19 8037.86
8.39 7731.44
16.78 5977.99
33.55 7176.65
67.11 6826.94
134.22 6549.12
268.44 6686.81
536.87 6839.56
1073.74 6905.00
2147.48 5849.59

Memory partial write bandwidth
0.000512 36130.22
0.001024 36126.03
0.002048 36127.90
0.004096 36123.43
0.008192 36125.84
0.016384 36121.31
0.032768 34323.82
0.065536 21723.77
0.131072 22076.31
0.262144 21902.76
0.524288 13914.93
1.05 13884.65
2.10 13824.09
4.19 13792.28
8.39 11268.52
16.78 7086.97
33.55 6924.86
67.11 6956.45
134.22 6751.39
268.44 6890.73
536.87 7249.92
1073.74 7193.62
2147.48 6125.88

Memory partial read/write bandwidth
0.000512 11678.58
0.001024 11796.97
0.002048 11887.52
0.004096 11902.71
0.008192 11917.54
0.016384 11925.12
0.032768 11912.93
0.065536 11413.17
0.131072 11429.73
0.262144 11405.94
0.524288 10285.76
1.05 10252.24
2.10 10252.24
4.19 10278.36
8.39 9049.20
16.78 6688.59
33.55 5991.86
67.11 6004.73
134.22 6306.34
268.44 6266.59
536.87 6385.92
1073.74 6541.14
2147.48 5364.66



"size=0k ovr=0.82
2 3.87
4 3.84
8 2.81
16 3.75
24 3.94
32 3.90
64 3.98
96 3.99

"size=4k ovr=1.27
2 3.85
4 2.65
8 2.36
16 2.98
24 3.55
32 3.43
64 3.11
96 3.22

"size=8k ovr=1.73
2 3.93
4 2.98
8 2.36
16 3.33
24 3.79
32 3.50
64 3.47
96 3.38

"size=16k ovr=2.64
2 3.86
4 2.67
8 2.43
16 3.38
24 3.93
32 4.36
64 4.18
96 4.34

"size=32k ovr=4.52
2 3.87
4 2.87
8 2.85
16 3.61
24 3.65
32 5.15
64 6.33
96 6.33

"size=64k ovr=8.15
2 3.88
4 2.40
8 3.05
16 3.71
24 5.75
32 6.68
64 8.72
96 8.49

Usage: tlb [-c] [-L <line size>] [-M len[K|M]] [-W <warmup>] [-N <repetitions>]

Memory load parallelism
Usage: par_mem [-L <line size>] [-M len[K|M]] [-W <warmup>] [-N <repetitions>]

STREAM copy latency: inf nanoseconds
STREAM copy bandwidth: 0.00 MB/sec
STREAM scale latency: inf nanoseconds
STREAM scale bandwidth: 0.00 MB/sec
STREAM add latency: inf nanoseconds
STREAM add bandwidth: 0.00 MB/sec
STREAM triad latency: inf nanoseconds
STREAM triad bandwidth: 0.00 MB/sec
STREAM2 fill latency: inf nanoseconds
STREAM2 fill bandwidth: 0.00 MB/sec
STREAM2 copy latency: inf nanoseconds
STREAM2 copy bandwidth: 0.00 MB/sec
STREAM2 daxpy latency: inf nanoseconds
STREAM2 daxpy bandwidth: 0.00 MB/sec
STREAM2 sum latency: inf nanoseconds
STREAM2 sum bandwidth: 0.00 MB/sec

Memory load latency
"stride=16
0.00049 1.772
0.00098 1.772
0.00195 1.772
0.00293 1.772
0.00391 1.772
0.00586 1.772
0.00781 1.772
0.01172 1.772
0.01562 1.772
0.02344 1.772
0.03125 1.773
0.04688 2.076
0.06250 2.065
0.09375 2.065
0.12500 2.065
0.18750 2.065
0.25000 2.069
0.37500 2.310
0.50000 2.310
0.75000 2.310
1.00000 2.310
1.50000 2.310
2.00000 2.311
3.00000 2.276
4.00000 2.285
6.00000 2.277
8.00000 2.320
12.00000 3.691
16.00000 4.079
24.00000 3.370
32.00000 3.722
48.00000 3.702
64.00000 3.770
96.00000 3.684
128.00000 3.692
192.00000 3.701
256.00000 3.679
384.00000 3.658
512.00000 3.580
768.00000 3.519
1024.00000 3.514
1536.00000 3.732
2048.00000 4.043

"stride=32
0.00049 1.772
0.00098 1.772
0.00195 1.772
0.00293 1.772
0.00391 1.772
0.00586 1.772
0.00781 1.772
0.01172 1.772
0.01562 1.772
0.02344 1.772
0.03125 1.774
0.04688 3.100
0.06250 3.100
0.09375 3.100
0.12500 3.100
0.18750 3.105
0.25000 3.119
0.37500 3.587
0.50000 3.587
0.75000 3.587
1.00000 3.587
1.50000 3.587
2.00000 3.590
3.00000 3.575
4.00000 3.581
6.00000 3.574
8.00000 3.706
12.00000 4.919
16.00000 5.488
24.00000 5.489
32.00000 5.567
48.00000 5.152
64.00000 5.615
96.00000 5.315
128.00000 5.482
192.00000 5.426
256.00000 5.312
384.00000 5.144
512.00000 5.178
768.00000 5.070
1024.00000 5.064
1536.00000 7.037
2048.00000 4.939

"stride=64
0.00049 1.771
0.00098 1.771
0.00195 1.771
0.00293 1.771
0.00391 1.771
0.00586 1.771
0.00781 1.772
0.01172 1.772
0.01562 1.771
0.02344 1.771
0.03125 1.774
0.04688 4.429
0.06250 4.428
0.09375 4.429
0.12500 4.440
0.18750 4.460
0.25000 4.514
0.37500 5.423
0.50000 5.421
0.75000 5.422
1.00000 5.418
1.50000 5.418
2.00000 5.428
3.00000 5.398
4.00000 5.416
6.00000 5.412
8.00000 5.672
12.00000 11.203
16.00000 11.428
24.00000 9.893
32.00000 10.031
48.00000 10.029
64.00000 10.492
96.00000 10.420
128.00000 10.141
192.00000 10.443
256.00000 10.084
384.00000 10.058
512.00000 9.337
768.00000 9.339
1024.00000 9.334
1536.00000 9.095
2048.00000 8.987

"stride=128
0.00049 1.772
0.00098 1.771
0.00195 1.772
0.00293 1.772
0.00391 1.771
0.00586 1.772
0.00781 1.772
0.01172 1.771
0.01562 1.772
0.02344 1.772
0.03125 1.773
0.04688 4.180
0.06250 4.429
0.09375 4.429
0.12500 4.429
0.18750 4.476
0.25000 4.560
0.37500 9.449
0.50000 9.449
0.75000 9.448
1.00000 9.450
1.50000 9.449
2.00000 9.451
3.00000 9.466
4.00000 9.470
6.00000 9.467
8.00000 10.739
12.00000 30.578
16.00000 32.551
24.00000 30.186
32.00000 28.482
48.00000 31.817
64.00000 31.609
96.00000 29.478
128.00000 31.579
192.00000 31.010
256.00000 32.188
384.00000 29.372
512.00000 29.725
768.00000 29.031
1024.00000 28.926
1536.00000 28.405
2048.00000 28.402

"stride=256
0.00049 1.772
0.00098 1.772
0.00195 1.771
0.00293 1.771
0.00391 1.772
0.00586 1.772
0.00781 1.771
0.01172 1.771
0.01562 1.771
0.02344 1.771
0.03125 1.772
0.04688 4.428
0.06250 4.429
0.09375 4.429
0.12500 4.439
0.18750 4.451
0.25000 4.495
0.37500 16.239
0.50000 16.239
0.75000 16.239
1.00000 16.240
1.50000 16.241
2.00000 16.248
3.00000 16.085
4.00000 16.126
6.00000 16.088
8.00000 17.425
12.00000 68.129
16.00000 79.672
24.00000 75.712
32.00000 72.158
48.00000 78.575
64.00000 79.643
96.00000 76.677
128.00000 78.948
192.00000 78.628
256.00000 76.661
384.00000 74.165
512.00000 73.904
768.00000 72.147
1024.00000 72.161
1536.00000 70.519
2048.00000 69.875

"stride=512
0.00049 1.772
0.00098 1.772
0.00195 1.772
0.00293 1.772
0.00391 1.772
0.00586 1.771
0.00781 1.772
0.01172 1.771
0.01562 1.772
0.02344 1.772
0.03125 1.772
0.04688 4.429
0.06250 4.429
0.09375 4.429
0.12500 4.440
0.18750 4.446
0.25000 4.484
0.37500 16.940
0.50000 16.941
0.75000 16.940
1.00000 16.941
1.50000 16.938
2.00000 16.940
3.00000 16.689
4.00000 16.755
6.00000 16.692
8.00000 17.378
12.00000 92.348
16.00000 88.858
24.00000 93.311
32.00000 78.234
48.00000 94.483
64.00000 84.389
96.00000 86.403
128.00000 89.651
192.00000 88.133
256.00000 85.686
384.00000 84.081
512.00000 82.360
768.00000 79.832
1024.00000 79.812
1536.00000 78.128
2048.00000 77.143

"stride=1024
0.00098 1.771
0.00195 1.772
0.00293 1.771
0.00391 1.771
0.00586 1.771
0.00781 1.771
0.01172 1.772
0.01562 1.771
0.02344 1.771
0.03125 1.772
0.04688 4.454
0.06250 4.437
0.09375 4.428
0.12500 4.433
0.18750 4.438
0.25000 4.465
0.37500 17.254
0.50000 17.252
0.75000 17.251
1.00000 17.253
1.50000 17.254
2.00000 17.256
3.00000 16.792
4.00000 16.912
6.00000 16.797
8.00000 17.191
12.00000 83.781
16.00000 102.908
24.00000 84.246
32.00000 74.774
48.00000 92.389
64.00000 88.833
96.00000 82.419
128.00000 87.506
192.00000 85.898
256.00000 84.426
384.00000 81.824
512.00000 81.797
768.00000 79.908
1024.00000 79.817
1536.00000 78.044
2048.00000 77.290


Random load latency
"stride=16
0.00049 1.771
0.00098 1.772
0.00195 1.771
0.00293 1.771
0.00391 1.772
0.00586 1.771
0.00781 1.772
0.01172 1.771
0.01562 1.771
0.02344 1.772
0.03125 1.774
0.04688 4.429
0.06250 4.436
0.09375 5.101
0.12500 4.942
0.18750 5.110
0.25000 14.231
0.37500 18.542
0.50000 19.753
0.75000 20.127
1.00000 20.936
1.50000 22.964
2.00000 25.920
3.00000 28.871
4.00000 32.410
6.00000 50.000
8.00000 64.287
12.00000 106.251
16.00000 120.601
24.00000 111.766
32.00000 120.539
48.00000 130.325
64.00000 130.798
96.00000 131.621
128.00000 132.052
192.00000 116.934
256.00000 109.281
384.00000 121.757
512.00000 124.703
768.00000 127.122
1024.00000 142.690
1536.00000 158.135
2048.00000 139.742



[Mon May 3 22:11:15 UTC 2010]


More information about the freebsd-performance mailing list