gjournal performance issues

[Pawel Jakub Dawidek]

On Tue, Jun 26, 2007 at 12:37:02AM +0200, Fluffles wrote:
> Hello list,
> 
> I'm testing gjournal present in -CURRENT as of 13 June 2007. So far i'm not really impressed with performance, i'm writing to the list for any suggestions and information 
> regarding gjournal.
> 
> First, my setup:
> 8 disks in RAID5 using geom_raid5, gjournal on top where both the journal (1GB) and the data is stored on the same consumer. Since gjournal uses both metadata and file/dir 
> for journaling, this means that, theoretically, the write speed of sequential operations is doubled. Unfortunately, it appears to have crashed.
> 
> My problems:
> - first, throughput appears to be only 8% of the throughput when not using gjournal at all. Whereas it should be close to 50%.

I had numbers in my presentation somewhere, but for single thread doing
sequential write when journal and data are on the same disk (which is
worstcase scenario) I recall something between 40%-50% of what you have
from regular UFS.

> - second, during the 'switch' (writing the journal to its final location and starting a new journal) it appears no read operations are possible to the .journal device. If 
> the .journal is /usr, that means the whole system basically freezes for 3 to 5 seconds. Not really sexy. Why would it block read requests?

On journal switch all operations that use vn_start_write() are blocked.
Read operations shouldn't use vn_start_write(), but for example close
operation is using it. I've some patches to eliminate it, because it is
only needed when we close already deleted file and need to free its
blocks, etc. which is quite rare case.

> - when using one consumer for both journal and data, it appears the journal is placed at the end of the device. Why? Normally, the beginning of a disk is the fastest and 
> therefore preferable location for the journal.

To allow to mount file system without gjournal.

> - when analysing graid5 sysctl statistics, it appears gjournal is causing non-contiguous I/O which causes a lot of 2-phase I/O's (involving both reading and writing for 1 
> write request), the performance difficulties are most probably related to this issue. Why doesn't gjournal read a chunk of journal (50MB) and then write it? And why doesn't 
> it write contiguously?

Actually gjournal should be much better on this than UFS. It combines
contiguous I/Os up to MAXPHYS and eliminates multiple write in the same
place. On the other hand graid5 should be able to delay writes a bit to
see if it can do full stripe write.

> I've tried:
> - playing with graid5 tunables, including disabling write-back buffer
> - playing with journal tunables, including disabling optimization (combining), reducing parallel operations to 1, reducing journal switch time and more
> - kmem is 500MB, gjournal can use 250MB of kernel memory for it's cache (more than the default)
> - standard UFS2 using async option and without softupdates, newfs used with -J parameter
> 
> Anyone has any input? I was hoping for at least 40MB/s throughput and no blocking I/O for read requests.

Have you tried not to use graid5 with gjournal? Could you retest for raw
disk and/or gmirror and/or graid3?

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-geom/attachments/20070626/9792105a/attachment.pgp