9-stable from i386 to amd64

[Konstantin Belousov]

On Sat, Feb 11, 2012 at 12:02:07AM -0600, Dan Nelson wrote:
> In the last episode (Feb 10), Randy Bush said:
> > is there a recipe for moving from i386 to amd64?
> > 
> > on a very remote system, i made the migration from 7.4 to 8.2 to 9.0, all
> > 32-bit.  it was done with repeated
> > 
> >     make buildworld
> >     make kernel.new [0]
> >     nextboot -k kernel.new
> >     reboot
> >     make installworld
> >     etc
> > 
> >     [0] - well, there were some mv(1)s in there :)
> > 
> > so after it was happy with 9.0 i386, i went to move to amd64 with
> > 
> >     make buildworld TARGET=amd64
> >     make kernel TARGET=amd64 DESTDIR=kernel.new [0]
> >     nextboot -k kernel.new
> >     reboot
> > 
> > it did not come back from the reboot, and required a manual reset.  i have
> > no console access to the machine, not my choice.
> > 
> > clue bat please.
> 
> You probably got bit by a mismatched /libexec/ld-elf.so. The kernel expects
> that to be the "native" version, and on a 64-bit kernel it also expects a
> ld-elf32.so to be the "compat" 32-bit version.  When you rebooted onto the
> 64-bit kernel, it couldn't find /libexec/ld-elf32.so to run any of the
> 32-bit binaries on the system.  My guess is that your reboot attempt died in
> /sbin/init, prompting for a path to /bin/sh.  If you compiled with a static
> /bin/sh for performance, it probably died very early in /etc/rc.
These statements are false, esp. worrying is that they are
interwinned with some facts that get tilted to support false presumption.

Kernel do not care about which interpreter is /libexec/ld-elf.so.
The path to the interpreter is specified in the binary itself. So if you
have 32bit binary that put '/libexec/ld-elf.so.1' into PH_INTERP,
and /libexec/ld-elf.so.1 is 32bit, then amd64 kernel properly executes
that combination.

Kernel has a hack that falls back to try to use /libexec/ld-elf32.so.1
for some 'brands' of ELF images, in particular, for 32bit binaries. This
is done to help in situation when 32bit binaries also specified the
same path for interpreter.

If you have 32bit world installed and booted 64bit kernel, it will boot. 
It is the same as running 32bit world in the jail.
The management functions, like configuring network interfaces, ZFS
and many other system setup functionality does not work, indeed.
> 
> I think copying ld-elf.so over to ld-elf32.so might have been all you needed
> to boot, but that would end up with a 64-bit kernel running a true 32-bit
> userland with all the libraries in the "wrong" place, and your
> "installworld" step would replace them with their 64-bit equivalents and
> your install would die halfway through, leaving you with a large mess to
> clean up.
Absolute false.

> 
> The cleanest upgrade path is to prepare your 32-bit root to be bootable by
> both 32- and 64-bit kernels: copy the ld-elf32.so that was built during your
> buildworld over to /libexec/ld-elf32.so, and also make copies of
> /lib and /usr/lib to /lib32 and /usr/lib32 respectively.  That way when you
> reboot to a 64-bit kernel, your 32-bit executables will be running
> "correctly" out of compat32 paths and your installworld should succeed.
> 
> When I did all this on a local system, I made judicious use of ZFS snapshots
> and clones, preserving a bootable clone of my original system plus
> intermediate versions all the way until I was happy with the result.  I've
> never done it completely remotely, but if you do a trial run or two on a
> local machine or VM, you should be able to it confidently remotely.
> 
> -- 
> 	Dan Nelson
> 	dnelson at allantgroup.com
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20120211/a44a73f9/attachment.pgp