Using the ISO releases on USB sticks.

Mon Sep 15 05:12:53 UTC 2014

El día Sunday, September 14, 2014 a las 01:55:44PM -0700, Mehmet Erol Sanliturk escribió:

> On Sun, Sep 14, 2014 at 12:24 PM, Matthias Apitz <guru at unixarea.de> wrote:
> 
> > I always test new versions of FreeBSD on USB keys, right now while
> > typing I run a 11-CURRENT with KDE ... and mutt as MUA booted from an USB
> > key, while the disk in the laptop has still an older FreeBSD.
> >
> > The procedure to creat a boot-able USB key is simple:
> >
> > - make a file system on it
> > - mount it to /mnt
> > - do 'make installkernel' and 'make installworld' with DESTDIR=/mnt
> > - enrich the file system in /mnt with some stuff (for example install
> >   packages into it with 'chroot /mnt pkg install ...')
> > - unmount and reboot
> >
> > even better, if you do not use an USB key directly, but do all the above
> > to a mdconfig file system (a file system in a normal plain file) and
> > copy later the result with dd(1) to the real USB key;
> >
> > let me know if you need a detailed procedure of it.
> >
> 
> Dear Matthias ,
> 
> 
> If you supply such an algorithm , me and other people may use it to
> generate boot-able USB sticks with some installed packages .
> 
> Thank you very much .
> 
> 
> Mehmet Erol Sanliturk

here it comes; comments are always welcome;

	matthias

$Id: buildingImage.txt,v 1.15 2014/08/15 17:39:21 guru Exp $

       Building bootable FreeBSD/i386 USB keys 
             <guru at unixarea.de>

This small howto file describes how to build and enrich bootable USB keys.
Some ideas are stolen from:
http://bsdimp.blogspot.com/2007/10/building-bootable-freebsdi386-images.html

File operations, esp. of a large amount of small files, are very SLOW
on USB keys. That's why we use in general the following procedure:

1. dd(1) the complete /dev/da0 to a plain file, an image of the USB key;
2. use mdconfig(8) to make a memory disk of this file and mount(8) it;
3. do everything of the formatting and installation in this virtual disk;
4. unmount(8) it and dd(1) it back to the USB key;

here we go (use for /dev/da0 what your key is in /var/log/messages):

check the size of your USB key with:

# dd if=/dev/da0 of=/dev/null bs=8m
1878+1 records in
1878+1 records out
15758000128 bytes transferred in 783.312179 secs (20117139 bytes/sec)

note: this step is essential to define an image which later fits again to the
real USB key; my "16 Gbyte" keys vary between 1868 and 1914 blocks of 8m
(which is ofc lesse than 16 GByte); that's why I decided to use always the
number 1860 for my keys of this "16 marketing GBytes".

now we create an empty large disk file of this size 1860 blocks of 8m:

# dd if=/dev/zero of=da0.r269739 bs=8m count=1860

attach the file da0.r269739 as a memory disk:

# mdconfig -a -t vnode -f da0.r269739
md0

now we use the normal procedure to create a bootable UFS on /dev/md0:

# fdisk -I md0
# fdisk -B md0
# bsdlabel -w md0s1 auto
# bsdlabel -B md0s1
# bsdlabel -e md0s1  # edit the disk label and change partition "a" from "unused" to "4.2BSD"

build the file system:

# newfs /dev/md0s1a
# mount /dev/md0s1a /mnt

note: if your running host system is older than 20121201 (see src/UPDATING) you
must add some new user if you try to install 'head':

# pw useradd -n auditdistd -g audit -c "Auditdistd unprivileged user" -d /var/empty -s /usr/sbin/nologin

install the system from your host which must have been compiled
in /usr/src; if you compiled it in another place check for MAKEOBJDIRPREFIX
below;

# cd /usr/src
# make installworld  DESTDIR=/mnt
# make installkernel DESTDIR=/mnt KERNCONF=GENERIC INSTALL_NODEBUG=t
# make distrib-dirs  DESTDIR=/mnt
# make distribution  DESTDIR=/mnt

the new system in /mnt needs some files to start:

# echo /dev/da0s1a / ufs rw,noatime 1 1 > /mnt/etc/fstab
# cp /etc/wpa_supplicant.conf /mnt/etc

# cat <<EOF > /mnt/etc/rc.conf
tmpmfs="YES"
tmpsize="128m"
wlans_ath0="wlan0"
ifconfig_wlan0="WPA DHCP"
ifconfig_bge0="DHCP"
hostname=tiny-r269739
sshd_enable="YES"
moused_enable="YES"
dbus_enable="YES"
hald_enable="YES"
vboxguest_enable="YES"
vboxservice_enable="YES"
keymap="german.iso"
# powerd
powerd_enable="YES"
powerd_flags="-a max -b adp"
performance_cx_lowest_cx_lowest="Cmax"
economy_cx_lowest="Cmax"
EOF

# cat <<EOF > /mnt/boot/loader.conf
kern.cam.boot_delay="10000"
kern.cam.scsi_delay="3000"
hint.p4tcc.0.disabled=1
hint.acpi_throttle.0.disabled=1
EOF

to use the above created USB key to install another system (the EeePC for
example, or the Acer) we need 'src' and 'obj' as well on the key:

# cd /usr
# cp -Rp src /mnt/usr
# cp -Rp obj /mnt/usr

but remove the SVN control files from the copied soure tree:

# rm -rf /mnt/usr/src/.svn

enrich the USB key with the prepared packages:

# cd /usr/local/poudriere/data/packages/freebsd-head-ports-head/All
# cp -p * /mnt/PKGDIR

o run later

# cd /usr/local/poudriere/data/packages/freebsd-head-ports-head/All
# rsync -avz . /mnt/PKGDIR

as well we need in 11-CURRENT now the new pkg(8) command; we compile
the old way ports-mgmt/pkg-devel and install only 'pkg-static' into
the /mnt system:

# cd /usr/ports/ports-mgmt/pkg-devel
# make install
# chroot /mnt mkdir -p /usr/local/sbin
# cp -p /usr/local/sbin/pkg-static /mnt/usr/local/sbin

let's see if this works:

# chroot /mnt /usr/local/sbin/pkg-static help

first we have to build the 'repo' from the directory of all
packages; we run:

# chroot /mnt pkg-static repo /PKGDIR
Creating repository in /PKGDIR: 83%
...
Packing files for repository: 100%

the recent version of pkg(8) needs in any case the definition
of the respository; we create the myrepo.conf file as:

# chroot /mnt mkdir -p /usr/local/etc/pkg/repos
# vim /mnt/usr/local/etc/pkg/repos/myrepo.conf
# cat /mnt/usr/local/etc/pkg/repos/myrepo.conf

           FreeBSD: {
               url: "file:/PKGDIR",
               enabled: true,
           }

now we are able to install any stuff (and its dependencies), here
as an example 'autoconf-2.69' and its dependencies:

# chroot /mnt pkg-static -R /usr/local/etc/pkg/repos/  install autoconf-2.69
Updating repository catalogue
FreeBSD repository is up-to-date
All repositories are up-to-date
The following 5 packages will be affected (of 0 checked):

New packages to be INSTALLED:
        autoconf: 2.69
        indexinfo: 0.2
        perl5: 5.16.3_11
        m4: 1.4.17_1,1
        autoconf-wrapper: 20131203

The process will require 45 MB more space
13 MB to be downloaded

Proceed with this action [y/N]: y
Fetching autoconf-2.69.txz: 100% of 529 KB                                   
Fetching indexinfo-0.2.txz: 100% of 6 KB                                       
Fetching perl5-5.16.3_11.txz: 100% of 12 MB                                   
Fetching m4-1.4.17_1,1.txz: 100% of 183 KB                                   
Fetching autoconf-wrapper-20131203.txz: 100% of 4 KB                           
Checking integrity... done (0 conflicting)
[1/5] Installing indexinfo-0.2: 100%
[2/5] Installing perl5-5.16.3_11: 100%
[3/5] Installing m4-1.4.17_1,1: 100%
[4/5] Installing autoconf-wrapper-20131203: 100%
[5/5] Installing autoconf-2.69: 100%

are they really there?:

# chroot /mnt pkg-static info                          
autoconf-2.69                  Automatically configure source code on many Un*x platforms
autoconf-wrapper-20131203      Wrapper script for GNU autoconf
indexinfo-0.2                  Utility to regenerate the GNU info page index
m4-1.4.17_1,1                  GNU m4
perl5-5.16.3_11                Practical Extraction and Report Language

later, when the system we building in /mnt is booted, we can use
pkg-static directly with the same repo, we do:

# pkg-static -R /usr/local/etc/pkg/repos/  install foo-x.x

copy some initial xorg.conf files to see what we can make use
of them later:

# cp /etc/X11/xorg.* /mnt/etc/X11

add an unpriv user 'guru' (for beeing able to SSH-in):

# chroot /mnt /usr/sbin/adduser

and enrich its HOME with files to bringup KDE:

# cp -p ~guru/.xinitrc   /mnt/home/guru
# cp -p ~guru/.xserverrc /mnt/home/guru

umount the key, detach the memory disk and copy the image back
with dd(1) to the USB device:

# umount /mnt
# mdconfig -d -u md0
# dd if=da0.r269739 of=/dev/da0 bs=8m

soem more hints for special cases:

If we have the real USB device locally, but the image on a laptop
without USB ports, one may use:

# ssh -C root at 10.49.8.36 dd of=/dev/da1 bs=8m < da0-r269739
0+823717 records in
1868+0 records out
15669919744 bytes transferred in 12135.626342 secs (1291233 bytes/sec)
#
(i.e. around 3h 17min)

To write the image from the VM to Ubuntu's USB port:

$ ssh -C 192.168.2.102
$ sudo chmod 0666 /dev/sdb
$ ssh -C 192.168.2.102 dd of=/dev/sdb bs=8M < da0-r269739

MAKEOBJDIRPREFIX hints to compile in and install from some other places:

If you want to build world and kernel out of SVN in 'src' and into 'obj', but in
some other place as /usr/[src|obj], for example in /home/guru/head/[src|obj]
use this procedure:

# mkdir -p /home/guru/head/obj
# MAKEOBJDIRPREFIX=/home/guru/head/obj ; export MAKEOBJDIRPREFIX
# cd /home/guru/head
# svn checkout svn://svn.freebsd.org/base/head src

# cd /home/guru/head/src
# make buildworld
# make buildkernel KERNCONF=GENERIC
...

if you have built the kernel and world in some other place as /usr/src:

# MAKEOBJDIRPREFIX=/home/guru/head/obj ; export MAKEOBJDIRPREFIX
# cd /home/guru/head/src
# make installworld  DESTDIR=/mnt
# make installkernel DESTDIR=/mnt KERNCONF=GENERIC INSTALL_NODEBUG=t
# make distrib-dirs  DESTDIR=/mnt
# make distribution  DESTDIR=/mnt
...

to use the key to install other system, you must have 'src' and 'obj'
again in the correct places; do:

# mkdir -p /mnt/usr/home/guru/head/
# cd /mnt
# ln -s usr/home home

# cd /home/guru/head/
# cp -Rp src /mnt/home/guru/head/    # 40 mins
# cp -Rp obj /mnt/home/guru/head/    # 20 mins

-- 
Matthias Apitz               |  /"\   ASCII Ribbon Campaign:
E-mail: guru at unixarea.de     |  \ /   - No HTML/RTF in E-mail
WWW: http://www.unixarea.de/ |   X    - No proprietary attachments
phone: +49-170-4527211       |  / \   - Respect for open standards
                             | en.wikipedia.org/wiki/ASCII_Ribbon_Campaign