GEOM profiling - how to?

[Lev Serebryakov]

Hello, Freebsd-geom.

  I'm  doing  some simple benchmarking of geom_raid5 in preparation of
  putting it into ports. And I notice strange results.

  It is array of 5 disks, stripsize=128k. All disks are SATA2 disks on
  ICH10R, AHCI driver (8.1-STABLE).

  Reading  from device itself (dd with bs=512K) gives speed of one HDD
  exactly.  gstat  shows  100% load of RAID geom and 1/5 of this speed
  (and 18-22% load) on all disk GEOMs.

  Reading big file  from   FS   (dd   with   bs=512k,   FS   block  size  32K,
  vfs.read_max=32)  gives  about  twice  speed  and every disk GEOM is
  loaded  38-42%.  CPU time is about 8% system, 0.5% interrupt, so CPU
  is not a bottle neck.

  Bigger block for "dd" doesn't wotk better.

  I  want  to know, why this configuration can not load disks up their
  top (and gives 4*HDD speed, about 320MB/s, speed).

  I  even  can not understand, why reeading from FS is beter than from
  raw device.

  Of  course,  I don't think you can me give answers for my questions.
  Really, I'm asking how could I peek into GEOM infrastructure to find
  bottle   necks,  understand  why  these  two  scenarios  gives  such
  different speed, etc.

  How could I profile I/O and GEOM?

-- 
// Black Lion AKA Lev Serebryakov <lev at FreeBSD.org>