misc/156933: ZFS receive after read on readonly=on filesystem is corrupted without warning

[D. Secret]

>Number:         156933
>Category:       misc
>Synopsis:       ZFS receive after read on readonly=on filesystem is corrupted without warning
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue May 10 17:20:10 UTC 2011
>Closed-Date:
>Last-Modified:
>Originator:     D. Secret
>Release:        8.2-RELEASE
>Organization:
>Environment:
FreeBSD z0.L93.com 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Thu Feb 17 02:41:51 UTC 2011     root at mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64

>Description:
If you read from a readonly=on ZFS filesystem before receiving a ZFS -i send for new snapshots it will receive the new snapshot and have a corrupted filesystem(that ZFS scrub does not detect!)  The one file on the filesystem I've included will show this corruption.  It should simply be a file of consecutive numbers, yet the last like will have some messed up data.

The ZFS 'images' were created as such:
13.zfs: A snapshot export of the filesystem
 -=- Append "14" to the replicate.dat file
 -=- Create snapshot @14
diff.zfs : a -i snapshot between @13 and @14.  It properly re-creates the file assuming you haven't read from the filesystem.  Note: Reading from the filesystem in this manner is present in some FAQ's about ZFS replication.


>How-To-Repeat:
1. Create a new zfs filesystem
zfs set readonly=on (my new filesystem)
cat zimg/13.zfs | zfs receive -F (my new filesystem)

OPTIONAL: cat (my new filesystem)/replicate.dat    
[This will cause the corruption if you do it]

cat zimg/diff.zfs | zfs receive (my new filesystem)

Now inspect (my new filesystem)/replicate.dat .. 

BUG file SHA1 is b3af2ae0f05cceb7efbcfc747750f56f53e4a69e
Proper file SHA1 is 42f9e9bf4534b43b57faaed39194115bf3517898

You'll notice it'll be different depending on if you performed the "optional" step.  I'm pretty sure this isn't proper.  I've verified that the atime of the file isn't changing, but there must be some subtle change going on that isn't being caught.  Also the zfs receive is allowed in both cases, but in one case the file is corrupted (vi shows 3x ^@ at the end, rather than the expected 14\n)

I can offer you access to a machine if you have difficulty reproducing this issue.  My scripts & sample filesystems are at:
http://sov.L93.com/bug.tgz
SHA1: 62b700d95096474ed7f3ae0b9249f9ec77e9a747
MD5: 9b9aa38b322cedd3e1223fc50f18a55b

There are no executables, just a very simple 5 or so line (rather basic) script to automate things if you choose to use them.  Please note it will destroy filesystem "z0/rdata_rcv" if you happen to have a ZFS filesystem by that name. ;)

A shar archive of the 11k TGZ is 711k, beyond what i can attach to this.  Hopefully the web link suffices.

>Fix:
I do not have a fix, but I do have what I believe is a 100% reproduction so I imagine a fix shouldn't be too difficult.   I would look for what change is being made to the ZFS filesystem while readonly=on is set and a file is read...It should be none, as far as I know.

>Release-Note:
>Audit-Trail:
>Unformatted: