FreeBSD 5.3; howto migrate from i386 to amd64 mode

Peter Wemm peter at
Wed Dec 15 08:52:34 PST 2004

On Wednesday 15 December 2004 12:20 am, Meyendriesch, Burkard wrote:
> hello folks,
> about a year ago I installed 5-CURRENT on my Athlon64 based box (Asus
> K8V). Because at that time amd64 mode didn't work very well with a
> lot of ports I installed FreeBSD in i386 mode. Meanwhile I followed
> the release path to 5.3. Everything works very fine (with i386 mode).
> Now I want to use the full power of my 64 bit CPU. I cross compiled
> the kernel to amd64 mode:
> # cd /usr/src
> # make kernel TARGET=amd64 TARGET_ARCH=amd64 KERNCONF=REINEKE64
> and tried to reboot the machine with the new kernel. The kernel
> himself seems to work but when he tries to mount the root filesystem
> the system hangs. I think something in my migration is wrong.
> Can someone please tell me the correct procedure to get my system
> work in amd64 mode without complete re-installation.
> Thanks a lot
> Burkard

There are ways, but it is painful.  You do not want to do what I'm going 
to post below..  Really. :-)

1) buildworld/buildkernel
2) turn off swap  (either swapoff or reboot with no swap)
3) newfs /dev/ad0s1b (or wherever)
4) mount /dev/ad0s1b /mnt
5) cd /usr/src; make installworld DESTDIR=/mnt
6) cd /usr/src/etc;  make distribution DESTDIR=/mnt
7) cp /etc/fstab /mnt/etc/fstab
8) copy your 64 bit kernel to /boot/kernel.amd64 or wherever you like
9) reboot
10) break into loader prompt
11) load /boot/kernel.amd64
12) set vfs.root.mountfrom="ufs:/dev/ad0s1b"
13) boot -s
14) now you're in single user on a small mini-install in your swap
15) mount your original file systems under /mnt.  all of them.  you'll 
have to do this part by hand.  read the /etc/fstab for the list.
16) up till this point, you are not committed.  #17 is the point of no 
17) you will have to mess with symlinks to make this work. but what you 
want is:
  cd /usr/src; make installworld DESTDIR=/mnt
  cd /usr/src; make installkernel DESTDIR=/mnt
you might need to use 'make -k' to get this to run through completely.
The problem is that you built with srcdir=/usr/src and objdir=/usr/obj.  
So the obj files were in /usr/obj/usr/src/... and now they're 
in /mnt/usr/obj/usr/src.  Since the installworld is running 
from /mnt/usr/src, it will expect to find its obj files 
in /usr/obj/mnt/usr/src.  You can try a mkdir -p /usr/obj/mnt/usr and 
then symlink from /usr/obj/mnt/usr/src -> /mnt/usr/obj/usr/src.  
Experimentation will be needed.
18) reboot
19) make sure you're in the 64 bit kernel and boot.  Cross fingers and 
everything else that can be crossed.
20) just to be safe, do another installworld while in 64 bit mode in 
case something got skipped due to 'make -k' mode.

Warning: I probably forgot some critical points.  I did this sort of 
thing a few times about 6-12 months ago and my instructions above are 
entirely from memory.  If you try it, you will need to be able to work 
your way out of the mess if you get stuck.

If you try it, you'd better have the reinstall CD handy.  You might 
still need it.  You can hose yourself.

If anybody would like to try a 32->64 transition on a fresh i386 install 
and take proper notes for a howto or a walkthrough, that might be a 
good thing.

Since you're presumably running on kernel.old right now, make sure you 
save it somewhere safe that you can get to.

The reason why it hangs are for many reasons.
1) our syscall translation is relatively primitive.  It works pretty 
well for 32 bit application level programs, but not for system 
programs.  eg: mount, init, etc.  sysctl is broken for 32 bit binaries 
because their idea of a 'long' doesn't match the kernel.  The mount 
interface has got binary blobs of data being passed around that we do 
not translate.
2) Right now, 32 bit binaries use the i386_userldt() facility in their 
early startup, which the amd64 kernel does not implement.  We have to 
use a custom 32-bit build of and that implements 
the thread-local-storage features via mechanisms that are available.  
So what is most likely happening is that /sbin/init is hanging or 
locking up, and as a result you see nothing.
3) the 32 bit syscall layer isn't bug free.  I've run into some apps 
that should work but don't.  Obviously things are not right yet.

Peter Wemm - peter at; peter at; peter at
"All of this is for nothing if we don't go to the stars" - JMS/B5

More information about the freebsd-amd64 mailing list