Maybe confused about AMD64 / i386 compatibility

[Joe Greco]

Background (most probably not necessary):

We use rsync snapshots and cheap IDE disks as part of our backup strategy.
Recently, a backup server started getting near full, and with 1.5TB disks
being what they are, we shot to replace a 750GB IDE with a 1.5TB SATA.

The backup server was upgraded to FreeBSD 7.2R/i386, it sports a 4GB flash
which leaves two drive bays in the Intel ISP1100 chassis available. 
Adding a SIL3112A gives us the SATA.  Good to go.  Create 1.5TB filesys,
and copy the old 750GB to it.

Option 1, using dd followed by growfs, doesn't work.  Groan.  growfs bombs
out with a negative integer in some array, and it isn't clear to me what
is happening.  So let's do the old slow but reliable dump/restore.

Except that a dump/restore of the filesystem, which is full of hardlinks,
causes massive resource consumption and eventual exhaustion.  Okay, fine.

Option 2, dump with gzip -1 to a file on the new drive (it'll have enough
space to restore in a separate step).  This works great.  Verify file
integrity (~400GB).  Also just fine.

Restore.  Watch restore grow to near 3GB and then bail.  Okay, obviously
an i386 architecture won't handle this restore.  So, think, aha, I'll go
and put it on a FreeBSD 7.1R/amd64 Opteron box with 8GB that is idle at
the moment.

-->> And here's where it all goes wonky.

Using a USB-to-SATA adapter, I plug in the drive to the amd64.  It sees
it.  I mount it.  All looks fine.  Good!

	# mount /dev/da1s1d /mnt
	# df
	Filesystem  1K-blocks       Used     Avail Capacity  Mounted on
	[...]
	/dev/da1s1d 1419039310 432472500 873043666    33%    /mnt

Now I try a restore.  So I do "gunzip < ad3s1e.dmp2 | restore rvf -" ...
and gzip claims the data isn't compressed.

	# gunzip < ad3s1e.dmp2 | restore rvf -
	gunzip: stdin: not in gzip format
	Verify tape and initialize maps
	End-of-tape encountered
	Tape is not a dump tape

Look at the data, looks generally compressed.  But file(1) sees it
as "data", not gzip data.

So, scratch head, wonder if it might be some odd gzip incompatibility,
get latest GNU gzip, still no go.  Rather annoyed, I think, "but I know
I verified this was a good file" - so I take the disk back to the i386,
put it in on the SATA controller, and it works fine.

So suddenly I think the USB-to-SATA is garbage and is corrupting the
data.  I md5 the first portions of the file, and go back, put the disk
on the amd64 with the USB-to-SATA.

	# dd if=ad3s1e.dmp2 bs=2048 count=1 | md5 

Returns f86ea62b2c77c58691001371c283c7a0 on i386,
	43ca53ad03650bc8c8fd1279ce19a675 on amd64.

!!!!!!!

This seems really odd.  So just to be sure, I go back to the i386 box,
this time with the USB-to-SATA...  and it returns the correct answer.

Double "!!!!!!!!!"

Now I finally get smart and try working with the raw disk device. 

	# dd if=/dev/da1s1 bs=2048 count=1024 | md5
	1024+0 records in
	1024+0 records out
	2097152 bytes transferred in 7.189189 secs (291709 bytes/sec)
	504efcdd93164c496c07603ba9b3774c

In all cases:

7.1amd64/USB:	504efcdd93164c496c07603ba9b3774c
7.2i386/SATA:	504efcdd93164c496c07603ba9b3774c
7.1amd64/USB:	504efcdd93164c496c07603ba9b3774c

So the underlying hardware and driver support "seems" to be fine.

And now this brings me to my question.

In my experience, FreeBSD filesystems are portable across i386-amd64.

Are there circumstances where this isn't the case?

I'm happy to concede that the filesystem in question is rather largish
(1.5TB) and probably stresses things a little bit.

# ls -al
total 432471268
drwxr-xr-x    5 root  wheel              512 Jun 13 11:07 .
drwxr-xr-x   24 root  wheel              512 Feb 18 09:18 ..
drwxrwxr-x    2 root  operator           512 Jun  5 08:20 .snap
-rw-r--r--    1 root  wheel     434044319653 Jun 10 19:25 ad3s1e.dmp2
-rw-r--r--    1 root  wheel               33 Jun 13 11:07 c1
-rw-r--r--    1 root  wheel               33 Jun 13 11:07 c1024
-rw-r--r--    1 root  wheel               33 Jun 13 11:07 c256
-rw-r--r--    1 root  wheel               33 Jun 13 11:07 c4
-rw-r--r--    1 root  wheel               80 Jun 13 10:43 logfile
drwx------  293 root  wheel            18432 Jun  8 04:25 lost+found
-rw-rw-rw-    1 root  wheel       8589934592 Jun  8 00:33 swapfile
drwxrwxrwx    9 root  wheel              512 Jun  8 14:14 x

Oh, and just to add some pain and agony to the puzzle ... it all seems
to work fine on 6.1-RELEASE/amd64 ... and 7.0-RELEASE/amd64.  But on a
different machine.  Which is where I'm doing my extraction.

So I'm not sure what to make of this.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.