FreeBSD 9.1 vs CentOS 6.3

Davide D'Amico davide.damico at contactlab.com
Fri Mar 22 09:03:37 UTC 2013


Hi, I'm doing performance tests on a DELL R720, follows dmesg:

Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.1-RELEASE #0: Tue Mar 19 10:24:21 CET 2013
     root:/usr/obj/usr/src/sys/R720 amd64
CPU: Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz (2500.05-MHz K8-class CPU)
   Origin = "GenuineIntel"  Id = 0x206d7  Family = 6  Model = 2d 
Stepping = 7
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x1fbee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX>
   AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
   AMD Features2=0x1<LAHF>
   TSC: P-state invariant, performance statistics
real memory  = 34359738368 (32768 MB)
avail memory = 33027436544 (31497 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <DELL   PE_SC3  >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 2 package(s) x 2 core(s) x 2 SMT threads
  cpu0 (BSP): APIC ID:  0
  cpu1 (AP): APIC ID:  1
  cpu2 (AP): APIC ID: 10
  cpu3 (AP): APIC ID: 11
  cpu4 (AP): APIC ID: 32
  cpu5 (AP): APIC ID: 33
  cpu6 (AP): APIC ID: 42
  cpu7 (AP): APIC ID: 43
ioapic1: Changing APIC ID to 1
ioapic2: Changing APIC ID to 2
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 32-55 on motherboard
ioapic2 <Version 2.0> irqs 64-87 on motherboard
kbd1 at kbdmux0
ctl: CAM Target Layer loaded
acpi0: <DELL PE_SC3> on motherboard
acpi0: Power Button (fixed)
acpi0: reservation of 0, 2000 (3) failed
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x7f irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x5f irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 53 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
igb0: <Intel(R) PRO/1000 Network Connection version - 2.3.4> port 
0xfc80-0xfc9f mem 0xdcb00000-0xdcbfffff,0xdcff0000-0xdcff3fff irq 38 at 
device 0.0 on pci1
igb0: Using MSIX interrupts with 9 vectors
igb0: Ethernet address: bc:30:5b:ef:27:b8
[...]
igb1: <Intel(R) PRO/1000 Network Connection version - 2.3.4> port 
0xfca0-0xfcbf mem 0xdcc00000-0xdccfffff,0xdcff4000-0xdcff7fff irq 37 at 
device 0.1 on pci1
igb1: Using MSIX interrupts with 9 vectors
igb1: Ethernet address: bc:30:5b:ef:27:b9
[...]
igb2: <Intel(R) PRO/1000 Network Connection version - 2.3.4> port 
0xfcc0-0xfcdf mem 0xdcd00000-0xdcdfffff,0xdcff8000-0xdcffbfff irq 36 at 
device 0.2 on pci1
igb2: Using MSIX interrupts with 9 vectors
igb2: Ethernet address: bc:30:5b:ef:27:ba
[...]
igb3: <Intel(R) PRO/1000 Network Connection version - 2.3.4> port 
0xfce0-0xfcff mem 0xdce00000-0xdcefffff,0xdcffc000-0xdcffffff irq 34 at 
device 0.3 on pci1
igb3: Using MSIX interrupts with 9 vectors
igb3: Ethernet address: bc:30:5b:ef:27:bb
[...]
pcib2: <ACPI PCI-PCI bridge> irq 53 at device 2.0 on pci0
pci3: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> irq 53 at device 2.2 on pci0
pci2: <ACPI PCI bus> on pcib3
mfi0: <Dell PERC H710P Mini (monolithics)> port 0xec00-0xecff mem 
0xddffc000-0xddffffff,0xddf80000-0xddfbffff irq 42 at device 0.0 on pci2
mfi0: Using MSI
mfi0: Megaraid SAS driver Ver 4.23
mfi0: MaxCmd = 3f0 MaxSgl = 46 state = b73c03f0
pcib4: <ACPI PCI-PCI bridge> irq 53 at device 3.0 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> irq 53 at device 3.2 on pci0
pci5: <ACPI PCI bus> on pcib5
pci0: <base peripheral> at device 5.0 (no driver attached)
pci0: <base peripheral> at device 5.2 (no driver attached)
pcib6: <PCI-PCI bridge> irq 16 at device 17.0 on pci0
pci6: <PCI bus> on pcib6
pci0: <simple comms> at device 22.0 (no driver attached)
pci0: <simple comms> at device 22.1 (no driver attached)
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xdf8fe000-0xdf8fe3ff irq 
23 at device 26.0 on pci0
usbus0: EHCI version 1.0
usbus0 on ehci0
pcib7: <ACPI PCI-PCI bridge> at device 28.0 on pci0
device_attach: pcib7 attach returned 6
pcib7: <ACPI PCI-PCI bridge> irq 19 at device 28.7 on pci0
pci8: <ACPI PCI bus> on pcib7
pcib8: <PCI-PCI bridge> at device 0.0 on pci8
pci9: <PCI bus> on pcib8
pcib9: <PCI-PCI bridge> at device 0.0 on pci9
pci10: <PCI bus> on pcib9
pcib10: <PCI-PCI bridge> at device 0.0 on pci10
pci11: <PCI bus> on pcib10
vgapci0: <VGA-compatible display> mem 
0xd8000000-0xd8ffffff,0xdeffc000-0xdeffffff,0xde000000-0xde7fffff irq 19 
at device 0.0 on pci11
pcib11: <PCI-PCI bridge> at device 1.0 on pci9
pci12: <PCI bus> on pcib11
ehci1: <EHCI (generic) USB 2.0 controller> mem 0xdf8ff000-0xdf8ff3ff irq 
22 at device 29.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci1
pcib12: <PCI-PCI bridge> at device 30.0 on pci0
pci13: <PCI bus> on pcib12
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
pcib13: <ACPI Host-PCI bridge> on acpi0
pci64: <ACPI PCI bus> on pcib13
pcib14: <ACPI PCI-PCI bridge> irq 85 at device 1.0 on pci64
pci65: <ACPI PCI bus> on pcib14
pcib15: <ACPI PCI-PCI bridge> irq 85 at device 2.0 on pci64
pci66: <ACPI PCI bus> on pcib15
pcib16: <ACPI PCI-PCI bridge> irq 85 at device 3.0 on pci64
pci67: <ACPI PCI bus> on pcib16
pcib17: <ACPI PCI-PCI bridge> irq 85 at device 3.2 on pci64
pci68: <ACPI PCI bus> on pcib17
pci64: <base peripheral> at device 5.0 (no driver attached)
pci64: <base peripheral> at device 5.2 (no driver attached)
pcib18: <ACPI Host-PCI bridge> on acpi0
pci63: <ACPI PCI bus> on pcib18
pci63: <base peripheral> at device 8.0 (no driver attached)
[...]
pci63: <dasp> at device 14.1 (no driver attached)
[...]
pci63: <dasp> at device 19.1 (no driver attached)
pci63: <dasp> at device 19.4 (no driver attached)
pci63: <dasp> at device 19.5 (no driver attached)
pci63: <base peripheral> at device 19.6 (no driver attached)
pcib19: <ACPI Host-PCI bridge> on acpi0
pci127: <ACPI PCI bus> on pcib19
[...]
pci127: <dasp> at device 14.1 (no driver attached)
[...]
pci127: <dasp> at device 19.1 (no driver attached)
pci127: <dasp> at device 19.4 (no driver attached)
pci127: <dasp> at device 19.5 (no driver attached)
pci127: <base peripheral> at device 19.6 (no driver attached)
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xec000-0xeffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 226d00001e00
device_attach: est0 attach returned 6
p4tcc0: <CPU Frequency Thermal Control> on cpu0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 226d00001e00
device_attach: est1 attach returned 6
p4tcc1: <CPU Frequency Thermal Control> on cpu1
est2: <Enhanced SpeedStep Frequency Control> on cpu2
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 226d00001e00
device_attach: est2 attach returned 6
p4tcc2: <CPU Frequency Thermal Control> on cpu2
est3: <Enhanced SpeedStep Frequency Control> on cpu3
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 224400001e00
device_attach: est3 attach returned 6
p4tcc3: <CPU Frequency Thermal Control> on cpu3
est4: <Enhanced SpeedStep Frequency Control> on cpu4
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 226d00001e00
device_attach: est4 attach returned 6
p4tcc4: <CPU Frequency Thermal Control> on cpu4
est5: <Enhanced SpeedStep Frequency Control> on cpu5
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 224400001e00
device_attach: est5 attach returned 6
p4tcc5: <CPU Frequency Thermal Control> on cpu5
est6: <Enhanced SpeedStep Frequency Control> on cpu6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 224400001e00
device_attach: est6 attach returned 6
p4tcc6: <CPU Frequency Thermal Control> on cpu6
est7: <Enhanced SpeedStep Frequency Control> on cpu7
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 224400001e00
device_attach: est7 attach returned 6
p4tcc7: <CPU Frequency Thermal Control> on cpu7
mfi0: 6040 (417120269s/0x0020/info) - Shutdown command received from host
mfi0: 6041 (boot + 4s/0x0020/info) - Firmware initialization started 
(PCI ID 005b/1000/1f34/1028)
mfi0: 6042 (boot + 4s/0x0020/info) - Firmware version 3.130.05-1796
mfi0: 6043 (boot + 5s/0x0008/info) - Battery Present
mfi0: 6044 (boot + 5s/0x0020/info) - Package version 21.1.0-0007
mfi0: 6045 (boot + 5s/0x0020/info) - Board Revision A01
mfi0: 6046 (boot + 6s/0x0008/info) - Battery temperature is normal
mfi0: 6047 (boot + 6s/0x0008/info) - Current capacity of the battery is 
above threshold
mfi0: 6048 (boot + 19s/0x0004/info) - Enclosure PD 20(c None/p1) 
communication restored
mfi0: 6049 (boot + 19s/0x0002/info) - Inserted: Encl PD 20
mfi0: 6050 (boot + 19s/0x0002/info) - Inserted: PD 20(c None/p1) Info: 
enclPd=20, scsiType=d, portMap=00, sasAddr=5e4ae020a26cb500,0000000000000000
mfi0: 6051 (boot + 19s/0x0002/info) - Inserted: PD 00(e0x20/s0)
mfi0: 6052 (boot + 19s/0x0002/info) - Inserted: PD 00(e0x20/s0) Info: 
enclPd=20, scsiType=0, portMap=04, sasAddr=5000c50054982339,0000000000000000
mfi0: 6053 (boot + 19s/0x0002/info) - Inserted: PD 01(e0x20/s1)
mfi0: 6054 (boot + 19s/0x0002/info) - Inserted: PD 01(e0x20/s1) Info: 
enclPd=20, scsiType=0, portMap=05, sasAddr=5000c50054986c91,0000000000000000
mfi0: 6055 (boot + 19s/0x0002/info) - Inserted: PD 02(e0x20/s2)
mfi0: 6056 (boot + 19s/0x0002/info) - Inserted: PD 02(e0x20/s2) Info: 
enclPd=20, scsiType=0, portMap=06, sasAddr=5000c50053da8f75,0000000000000000
mfi0: 6057 (boot + 19s/0x0002/info) - Inserted: PD 03(e0x20/s3)
mfi0: 6058 (boot + 19s/0x0002/info) - Inserted: PD 03(e0x20/s3) Info: 
enclPd=20, scsiType=0, portMap=07, sasAddr=5000c50053c57eb5,0000000000000000
mfi0: 6059 (boot + 19s/0x0002/info) - Inserted: PD 04(e0x20/s4)
mfi0: 6060 (boot + 19s/0x0002/info) - Inserted: PD 04(e0x20/s4) Info: 
enclPd=20, scsiType=0, portMap=00, sasAddr=5000c50053d364c5,0000000000000000
mfi0: 6061 (boot + 19s/0x0002/info) - Inserted: PD 05(e0x20/s5)
mfi0: 6062 (boot + 19s/0x0002/info) - Inserted: PD 05(e0x20/s5) Info: 
enclPd=20, scsiType=0, portMap=03, sasAddr=5000c50053da2b41,0000000000000000
mfi0: 6063 (boot + 19s/0x0002/info) - Inserted: PD 06(e0x20/s6)
mfi0: 6064 (boot + 19s/0x0002/info) - Inserted: PD 06(e0x20/s6) Info: 
enclPd=20, scsiType=0, portMap=02, sasAddr=4433221102000000,0000000000000000
mfi0: 6065 (boot + 19s/0x0002/WARN) - PD 06(e0x20/s6) is not a certified 
drive
mfi0: 6066 (boot + 19s/0x0002/info) - Inserted: PD 07(e0x20/s7)
mfi0: 6067 (boot + 19s/0x0002/info) - Inserted: PD 07(e0x20/s7) Info: 
enclPd=20, scsiType=0, portMap=01, sasAddr=4433221103000000,0000000000000000
mfi0: 6068 (boot + 19s/0x0002/WARN) - PD 07(e0x20/s7) is not a certified 
drive
mfi0: 6069 (417120312s/0x0020/info) - Time established as 03/20/13 
18:45:12; (32 seconds since power on)
mfi0: 6070 (417120445s/0x0020/info) - Host driver is loaded and operational
Timecounters tick every 1.000 msec
usbus0: 480Mbps High Speed USB v2.0
usbus1: 480Mbps High Speed USB v2.0
mfid0 on mfi0
mfid0: 285568MB (584843264 sectors) RAID volume 'BASE' is optimal
mfid1 on mfi0
mfid1: 121472MB (248774656 sectors) RAID volume 'L2ARC0' is optimal
mfid2 on mfi0
mfid2: 121472MB (248774656 sectors) RAID volume 'ZIL0' is optimal
mfid3 on mfi0
mfid3: 1143552MB (2341994496 sectors) RAID volume 'DATASTRIPE1' is optimal
mfid4 on mfi0
mfid4: 1143552MB (2341994496 sectors) RAID volume 'DATASTRIPE2' is optimal
ugen0.1: <Intel> at usbus0
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
ugen0.2: <vendor 0x8087> at usbus0
uhub2: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> 
on usbus0
ugen1.2: <vendor 0x8087> at usbus1
uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> 
on usbus1
uhub2: 6 ports with 6 removable, self powered
uhub3: 8 ports with 8 removable, self powered
ugen0.3: <no manufacturer> at usbus0
uhub4: <no manufacturer Gadget USB HUB, class 9/0, rev 2.00/0.00, addr 
3> on usbus0
SMP: AP CPU #1 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #4 Launched!
Timecounter "TSC-low" frequency 9765827 Hz quality 1000
Root mount waiting for: usbus0
uhub4: 6 ports with 6 removable, self powered
ugen0.4: <Avocent> at usbus0
ukbd0: <Keyboard> on usbus0
kbd0 at ukbd0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
ugen0.5: <Avocent> at usbus0
umass0: <SCSI Transparent Interface 0> on usbus0
Trying to mount root from ufs:/dev/mfid0p3 [rw]...
cd0 at umass-sim0 bus 0 scbus1 target 0 lun 0
cd0: <iDRAC Virtual CD 0329> Removable CD-ROM SCSI-0 device
cd0: 40.000MB/s transfers
cd0: Attempt to query device size failed: NOT READY, Medium not present
da0 at umass-sim0 bus 0 scbus1 target 0 lun 1
da0: <iDRAC Virtual Floppy 0329> Removable Direct Access SCSI-0 device
da0: 40.000MB/s transfers
da0: Attempt to query device size failed: NOT READY, Medium not present
ZFS filesystem version 5
ZFS storage pool version 28
ums0: <Mouse> on usbus0
ums0: 3 buttons and [Z] coordinates ID=0
ums1: <Mouse REL> on usbus0
ums1: 3 buttons and [XYZ] coordinates ID=0
igb0: link state changed to UP

With this loader.conf:
vm.kmem_size_max="64G"
vm.kmem_size="48G"
vfs.zfs.arc_min="4096M"
vfs.zfs.arc_max="15872M"
vfs.zfs.txg.synctime="1"
vfs.zfs.txg.timeout="5"
vfs.zfs.write_limit_override="0"
vfs.zfs.txg.write_limit_override="0"
vfs.zfs.zil_disable="0"

And with this sysctl.conf:
security.bsd.see_other_uids=0
net.inet.tcp.blackhole=2
net.inet.udp.blackhole=1
net.inet.tcp.sendspace=65536
net.inet.tcp.recvspace=65536
net.link.ether.inet.log_arp_wrong_iface=0
kern.ipc.somaxconn=32768
kern.threads.max_threads_per_proc=16384
kern.maxfiles=262144
kern.maxfilesperproc=131072
kern.ipc.nmbclusters=65536
kern.corefile="/var/coredumps/%U.%N.%P.core"

vfs.zfs.txg.timeout=5
vfs.zfs.l2arc_headroom=16
vfs.zfs.l2arc_write_max=26843546
vfs.zfs.l2arc_write_boost=26843546
vfs.zfs.vdev.min_pending=4
vfs.zfs.vdev.max_pending=64
vfs.zfs.l2arc_noprefetch=1
vfs.zfs.dedup.prefetch=0
vfs.zfs.prefetch_disable=1

I will use this server as a mysql-5.6 dbserver so I have a root 
partition using a hw raid1 and a /DATAZFS partition, follows configuration:

root# mfiutil show drives
mfi0 Physical Drives:
  0 (  279G) ONLINE <SEAGATE ST3300657SS ES65> SAS E1:S0
  1 (  279G) ONLINE <SEAGATE ST3300657SS ES65> SAS E1:S1
  2 (  558G) ONLINE <SEAGATE ST3600057SS ES65> SAS E1:S2
  3 (  558G) ONLINE <SEAGATE ST3600057SS ES65> SAS E1:S3
  4 (  558G) ONLINE <SEAGATE ST3600057SS ES65> SAS E1:S4
  5 (  558G) ONLINE <SEAGATE ST3600057SS ES65> SAS E1:S5
  6 (  119G) ONLINE <OCZ-VERTEX4 1.4> SATA E1:S6
  7 (  119G) ONLINE <OCZ-VERTEX4 1.4> SATA E1:S7
root# mfiutil show volumes
mfi0 Volumes:
   Id     Size    Level   Stripe  State   Cache   Name
  mfid0 (  278G) RAID-1      64k OPTIMAL Disabled <BASE>
  mfid1 (  118G) RAID-0      64k OPTIMAL Disabled <L2ARC0>
  mfid2 (  118G) RAID-0      64k OPTIMAL Disabled <ZIL0>
  mfid3 ( 1116G) RAID-1      64k OPTIMAL Disabled <DATA>

I have a /DATAZFS partition, too:
root# zfs get all DATAZFS
NAME     PROPERTY              VALUE                  SOURCE
DATAZFS  type                  filesystem             -
DATAZFS  creation              Thu Mar 21 10:45 2013  -
DATAZFS  used                  54.2G                  -
DATAZFS  available             1.02T                  -
DATAZFS  referenced            54.2G                  -
DATAZFS  compressratio         1.00x                  -
DATAZFS  mounted               yes                    -
DATAZFS  quota                 none                   default
DATAZFS  reservation           none                   default
DATAZFS  recordsize            16K                    local
DATAZFS  mountpoint            /DATAZFS               default
DATAZFS  sharenfs              off                    default
DATAZFS  checksum              on                     default
DATAZFS  compression           off                    default
DATAZFS  atime                 off                    local
DATAZFS  devices               on                     default
DATAZFS  exec                  on                     default
DATAZFS  setuid                on                     default
DATAZFS  readonly              off                    default
DATAZFS  jailed                off                    default
DATAZFS  snapdir               hidden                 default
DATAZFS  aclmode               discard                default
DATAZFS  aclinherit            restricted             default
DATAZFS  canmount              on                     default
DATAZFS  xattr                 off                    temporary
DATAZFS  copies                1                      default
DATAZFS  version               5                      -
DATAZFS  utf8only              off                    -
DATAZFS  normalization         none                   -
DATAZFS  casesensitivity       sensitive              -
DATAZFS  vscan                 off                    default
DATAZFS  nbmand                off                    default
DATAZFS  sharesmb              off                    default
DATAZFS  refquota              none                   default
DATAZFS  refreservation        none                   default
DATAZFS  primarycache          all                    default
DATAZFS  secondarycache        all                    default
DATAZFS  usedbysnapshots       0                      -
DATAZFS  usedbydataset         54.2G                  -
DATAZFS  usedbychildren        1.41M                  -
DATAZFS  usedbyrefreservation  0                      -
DATAZFS  logbias               latency                default
DATAZFS  dedup                 off                    local
DATAZFS  mlslabel                                     -
DATAZFS  sync                  standard               default
DATAZFS  refcompressratio      1.00x                  -
DATAZFS  written               54.2G                  -

root# zpool status -v
   pool: DATAZFS
  state: ONLINE
   scan: none requested
config:

NAME        STATE     READ WRITE CKSUM
DATAZFS     ONLINE       0     0     0
   mfid3     ONLINE       0     0     0
logs
   mfid2     ONLINE       0     0     0
cache
   mfid1     ONLINE       0     0     0

errors: No known data errors

root# mfiutil show firmware
mfi0 Firmware Package Version: 21.1.0-0007
mfi0 Firmware Images:
Name  Version                        Date         Time         Status
BIOS  5.31.01_4.12.05.00_0x05110000   6/20/2012
    6/20/2012
   active
CTLR  4.00-0014                      Aug 04 2011  12:49:17     active
PCLI  05.00-03:#%00008               Feb 17 2011  14:03:12     active
APP   3.130.05-1796                  Jul 27 2012  15:58:31     active
NVDT  2.1108.03-0094                 Jun 22 2012  12:40:55     active
BTBL  2.03.00.00-0004                Jul 27 2012  16:39:29     active
BOOT  06.253.57.219                  9/9/2010     15:32:25     active
root#

Then, I installed CentOS on the same hardware, same version of mysql 
(5.6.10-enterprise and 5.6.10-GA), and I tried an OLTP test (included in 
sysbench-0.5) executing 2000000 queries on 10000000 rows, with these 
results (transactions per seconds):

BSD 9.1 amd64 ZFS (without tweaks):
1 thread: 2780
2 threads: 4371
4 threads: 6876
8 threads: 8202
12 threads: 8077
16 threads: 7780
24 threads: 7563
32 threads: 7632
48 threads: 7960
64 threads: 8062

BSD 9.1 amd64 ZFS (with tweaks):
1 thread: 2589
2 threads: 4679
4 threads: 6438
8 threads: 7073
12 threads: 7121
16 threads: 7227
24 threads: 7132
32 threads: 7273
48 threads: 7623
64 threads: 7672

CentOS 6.3 (ext4, without tweaks):
1 thread: 4163
2 threads: 7653
4 threads: 10905
8 threads: 12511
12 threads: 13556
16 threads: 14832
24 threads: 16270
32 threads: 16733
48 threads: 16925
64 threads: 16895


Well, any of you could ask "why zfs, please try ufs that is better": you 
are right, but changing ZFS with UFS doesn't change so much:
http://lists.freebsd.org/pipermail/freebsd-fs/2013-March/016747.html

So I hope I'm missing something because the performances of a standard 
CentOS installation on the same hardware performs much better than a 9.1 
amd64 installation (I'm using the standard ULE scheduler included in the 
9.1 release).

Could anyone help me?

Thanks,
d.


More information about the freebsd-performance mailing list