compiling root filesystem into kernel (preferably tmpfs root filesystem)

[Travis Daygale]

David, thank you for the great information!  Yes, I would appreciate seeing the scripts and hearing about the other method you outline.  
Yes, you understand what I want to achieve exactly.
I see what you're saying about not needing to put it in the kernel file, though for a variety of reasons, I do prefer a single file in the end.
And I did find this after posting (didn't see it on previous searches, though I invested a lot of time looking before I posted):http://lists.freebsd.org/pipermail/freebsd-hackers/2006-November/018662.html
I have built a root image that I put in the kernel as described in the Nov 2006 post.  My UFS root image consists of /sbin/init, where init is a statically compiled C program that just spits out "Hello world" and sleeps, this binary runs fine under FBSD.  At this point, I have the kernel booting but it panics because it says it can't find init....  Hmmm...  I believe (haven't had time to test) that it is finding root?  Not sure though what loader args I might need to be providing?  Could it be a /dev issue (though I'm not needing sh, etc., since my init is not a real init)?  Still figuring this out.

Trever

--- On Sun, 4/5/09, David Naylor <naylor.b.david at gmail.com> wrote:

From: David Naylor <naylor.b.david at gmail.com>
Subject: Re: compiling root filesystem into kernel (preferably tmpfs root filesystem)
To: "Travis Daygale" <anti_spamsys at yahoo.com>
Cc: freebsd-hackers at freebsd.org
Date: Sunday, April 5, 2009, 1:14 PM

On Saturday 04 April 2009 21:52:14 Travis Daygale wrote:
> In both the loader and kernel compiling doc, I see snippets of information
> like this: #Make space in the kernel for a root filesystem on a md
> device.options MD_ROOT_SIZE=10 boot_dfltrootInstructs the kernel to mount
> the statically compiled-in root file system.

Yes, you can compile a fs image into the kernel.  This however will be static 
and if you want editing then will need to use unionfs with mdmfs.  tmpfs 
cannot be used for this as it does not yet (to my knowledge) support unionfs.  

> My question is, how does one compile a root filesystem into the FreeBSD
> kernel?  

Personally I wouldn't recommend taking the approach you want to do.  There is 
simply an easier way.  Just load the fs image as a kernel module (sort of).  
You get the same effect with more flexibility.  (I'll explain below).

> When mounted, I want this root filesystem to run entirely in 
> memory with no other backing store (not even a readonly flash disc nor
> other backing media such as DVD/CD). 

This is do-able.  I've created a CD that ejects it self when loaded 
completely.  (I thought it was cool :-))

> The standard FreeBSD DVD install disc 
> uses just such a root?  (Though seems to rely heavily on the rescue
> binaries being on a read only filesystem backed by the install DVD?) 

Can't comment, haven't used the FreeBSD CD/DVD's for years (since 6.0)

> I'm 
> still trying to reverse engineer how that was done, without much luck. Is
> there a place/documentation I should be finding?  PicoBSD, NanoBSD, NFS
> root diskless systems... all tantalizing close, but not the same thing
> (read only roots backed by media other than memory). The root filesystem
> I'm wanting would presumably be in some conceptual sense similar to
> initramfs in Linux land, if that helps explain what I'm trying to achieve.  

I'll give you a quick tutorial below (if you need further help please let me 
know).

>  In fact I have a Linux distribution which consists of a single giant
> kernel image and when boot, runs entirely in memory, the kernel in fact
> can't read filesystems other than tmpfs because no filesystems are compiled
> in.  

I think you are getting some concepts confused here.  

> It appears all of this won't be possible in FreeBSD (looks like ufs is 
> required) but it appears I can get close to this. Indeed, I'd love a way
> for the root filesystem in FreeBSD to be of type tmpfs, again similar to
> what is possible on the Linux side, though I'm much less concerned with the
> type of filesystem (it just needs to be compiled into the FreeBSD kernel
> and needs to be a memory backed filesystem when it mounts, no other backing
> store). Thanks in advance!

Ok, onto my explanation: my understanding is that you want to have some type 
of FreeBSD based system that is loaded completely into RAM.  Once loaded (at 
boot time) this system should have no reliance on any medium (other than 
RAM).  This system, once loaded, should behave the same as if it were backed 
by a hard drive (except the statefullness after reboots).  i.e. the 
filesystem should be editable.  

This is of course very possible.  


STAGE 1: The filesystem

In order to have the system in memory one needs a delivery method.  As 
mentioned before this is achieved using a MD device.  

MD's can have three types of backing, a vnode (aka file, on a CD/DVD or hard 
drive), or memory (purely in memory, AFAIK no swapping out) and swap (same as 
memory except my get swapped out).  Ignoring the subtle difference between 
memory and swap, swap is better.  Technically the forth is preload but this 
is the same as memory but done by the loader.  See md(4) for further details.  

Now, MD just imitates a hard drive, one still needs the data to put there.  
Any filesystem will suite this purpose.  My preference is UFS but ISO9660 
works just as easily (other options are not so easy but still do-able).  

Now, to create the filesystem, just install your system into a folder.  e.g.
# su -
# mkdir /tmp/world
# cd src; make world kernel distribution DESTDIR=/tmp/world
# cp /path/to/packages /tmp/world/tmp
# chroot /tmp/world sh -c 'cd  /tmp ; pkg_add *'
# rm -rf /tmp/world/tmp/*
# cat > /tmp/world/etc/fstab < _EOF
proc                    /proc           procfs  rw              0       0
tmpfs                   /tmp            tmpfs   rw              0       0
_EOF
* Now, edit /tmp/world as you require to make it work as you want (the easiest 
way is to create a Flash stick [as per my script], edit the system live and 
then copy all changes across).  *


STAGE 2: The filesystem image

*** For UFS ***
# makefs /tmp/world.ufs /tmp/world
# MDDEV=$(mdconfig -a -t vnode /tmp/world.ufs)
# tunefs -L ROOTFS /dev/$MDDEV
# mdconfig -d -u $MDDEV

*** For CD9660 ***
# mkisofs -quiet -sysid 
FREEBSD -rock -untranslated-filenames -max-iso9660-filenames -iso-level 
4 -volid DragonBSD -o $WORKDIR/DragonBSD.iso -volid 
DragonBSD -o /tmp/world.iso /tmp/world

Now, since these images are often much larger then required I prefer to 
compress them.  This allows more programs to be added to the image and it 
takes up less memory during runtime (not to mention faster load times).  
[I assume UFS option, do the appropriate for CD9660 option]

# mkuzip -s 8192 -o /tmp/world.uzip /tmp/world.ufs


STAGE 3: Loading the filesystem image

Now you have an image that can be loaded on boot, to do so add the following 
to loader.conf
# cd /path/to/boot/system/image
# cat >> boot/loader.conf < _EOF
rootfs_load="YES"
rootfs_type="mfs_root"
rootfs_name="/boot/world.uzip"
_EOF
# cp /tmp/world.uzip boot/

Now, to inform the system that you want it to boot off the memory system
# cat >> boot/loader.conf < _EOF
vfs.root.mountfrom="ufs:/dev/ufs/ROOTFS"
_EOF


STAGE 4: Making the Live System editable
Now, to make the whole system editable (everything) is quite the challenge and 
requires a change in the way the previous stages are done.  The concept is 
simple though.  First:

Because the filesystem was compressed (using mkuzip), it cannot be written to.  
If the system were not compressed and extra space was allocated to the UFS 
image then it can be editable.  Even the extra size at load time can be 
compensated for (since loader supports compressed modules [both gzip and 
bzip2] however you will be running the full image uncompressed in memory.  It 
is faster but much more expensive.  Just to give you an idea, I have gotten a 
700MB system to boot and run off a mini CD (210MB) and a system with 512MB of 
RAM, using the compressed approach with everything editable :-).  

To do this approach requires some changes to stage 2.  Basically, after 
completing the approach for UFS image do the following
# EXTRA_SIZE=32
# SIZE=$(($(du -m /tmp/world.ufs) + EXTRA_SIZE))
# dd if=/dev/zero of=/tmp/world.ufs count=$SIZE bs=1m  # NB, use zero to allow 
for compression
# MDDEV=$(mdconfig -a -t vnode /tmp/world.ufs)
# newfs -L ROOTFS -o space /dev/$MDDEV
# mkdir /tmp/btstrp
# mount /dev/$MDDEV /tmp/btstrp
# (cd /tmp/world; tar -cf - .) | (cd /tmp/btstrp; tar -xf -)
# umount /tmp/btstrp
# mdconfig -d -u $MDDEV

Next, DO NOT compress the image with mkuzip, instead do:
# gzip -9 /tmp/world.ufs

This requires either geom_uzip loaded or compiled into the kernel.

and, instead of the first part of stage 3 do
# cd /path/to/boot/system/image
# cat >> boot/loader.conf < _EOF
rootfs_load="YES"
rootfs_type="mfs_root"
rootfs_name="/boot/world.ufs"
_EOF
# cp /tmp/world.ufs.gz boot/

NOTE: this approach cannot be done using cd9660.

The second approach, the one I prefer requires a double boot image (one inside 
the other), where the one acts as a boot strap, mdconfig and mount's the 
embedded second image, creates a editable fs using mdmfs and unionfs it over 
the second image.  

This is done through using
# cat >> boot/loader.conf < _EOF
init_script="/chroot.sh"
init_chroot="/base"

where /chroot.sh basically does:
mount -o ro /dev/$(mdconfig -a -t vnode -o readonly -f /world.uzip).uzip /base
mdmfs -s 32m md /tmp
mount -t unionfs -o noatime -o copymode=transparent /tmp/base

It would be very nice to add unionfs support to tmpfs but not yet :-(.  The 
second approach I have not described fully, it is quite a bit more involved 
than the first but has great benefits, memory wise.  If you want more details 
about this approach please let me know.  

I've created a set of scripts that are designed to create LiveCD/DVD/Flash of 
FreeBSD.  There are three cd9660 images that it produces:
1) CD backed live system (using compressed ufs image)
2) Memory backed live system (using compressed ufs image)
3) CD backed live system (no compression).
And one Flash memory based image:
1) Flash based memory (using compressed ufs image and perpetual state overlay) 
[similar to option 1 above except the changes are permanent).  

If you would like access to these scripts please let me know and I will gladly 
forward them to you.  Also if you have any questions or want further 
clarification please ask.  

Regards,

David

Disclaimer: The commands may be incorrect but the procedure has been tried and 
tested.