gjournal failing to do its job

[Bartosz Fabianowski]

Good day

I recently set up a brand new 8-STABLE box with UFS2 + gjournal. After a 
hang and hard reboot, I noticed that the setup is not working. File 
systems came up inconsistent without the system even noticing. I had to 
manually run fsck in single user mode to get back to a consistent state.

The setup is nested as follows:

Drive:                        ada0
Slice (MBR partition):        ada0s2
geli full drive encryption:   ada0s2.eli
gjournal:                     ada0s2.eli.journal
bsdlabel:                     ada0s2.eli.journala - ada0s2.eli.journalg

Partitions a, d, e, f and g are UFS2 file systems. Partition b is swap. 
As the hard drive is fully encrypted, the system boots off a USB key 
containing the boot loader, kernel and fstab. In the fstab, file systems 
are addressed by their volume labels and swap by a label set with glabel.

As it appeared gjournal was working fine, I had set the gjournal flag on 
all UFS2 file systems and mounted them async. After the crash and 
reboot, gjournal reported "consistent" state. Then, a message informed 
me that / had not been unmounted cleanly and fsck ran on that one file 
system.  For the other file systems (located in partitions d, e, f and g 
above), I got no messages of any sort. However, once the system was up, 
they were clearly inconsistent.

A manual fsck run revealed serious inconsistencies such as unallocated 
inodes. The journal had clearly failed to do its job.

Could anyone point me to what I am doing wrong? I know I have nested 
several GEOM providers inside each other. I was under the impression 
this was the very concept behind GEOM. Is it that I cannot have a single 
journal with multiple bsdlabel partitions inside? Again, it would seem 
GEOM allows this - and I have found mailing list messages suggesting 
others have done this before.

On a related note, it would appear that there is no way to verify 
gjournal is actually active on a file system. I can set the journaling 
flag on any UFS2 partition - whether gjournal actually works or not. Is 
there some way to ensure that all file systems with journaling flag set 
are actually journaled?

The "gjournal list" command tells me that ada0s2.eli is a provider and 
ada0s2.eli.journal a consumer. But are all bsdlabel partitions inside 
the consumer covered as well? There seems to be no tool that can tell me 
that. I am hoping someone on the list can shed some light on this.

As a stop-gap measure, I have enabled softupdates on the file systems 
and removed the async flags from my fstab. I really would like to use 
journaling though - if I can find a way to enable it and make sure it is 
actually working.

Thanks,
- Bartosz Fabianowski