docs/62834: Diskless operation handbook section update
Jean-Francois Dockes
dockes at wanadoo.fr
Sat Feb 14 12:50:22 UTC 2004
>Number: 62834
>Category: docs
>Synopsis: Diskless operation handbook section update
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: freebsd-doc
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: change-request
>Submitter-Id: current-users
>Arrival-Date: Sat Feb 14 04:50:14 PST 2004
>Closed-Date:
>Last-Modified:
>Originator: Jean-Francois Dockes
>Release: FreeBSD 4.9-RELEASE-p2 i386
>Organization:
>Environment:
System: FreeBSD hautmedoc 4.9-RELEASE-p2 FreeBSD 4.9-RELEASE-p2 #0: Tue Feb 10 07:47:58 CET 2004 dockes at hautmedoc:/usr/obj/usr/src/sys/GENEDOC i386
>Description:
The diskless operation section needs to be updated, esp. because of 5.x changes
>How-To-Repeat:
>Fix:
Index: chapter.sgml
===================================================================
RCS file: /home/ncvs/doc/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml,v
retrieving revision 1.271
diff -u -r1.271 chapter.sgml
--- chapter.sgml 11 Feb 2004 19:28:25 -0000 1.271
+++ chapter.sgml 13 Feb 2004 08:54:01 -0000
@@ -2275,10 +2275,10 @@
<indexterm><primary>diskless operation</primary></indexterm>
<para>A FreeBSD machine can boot over the network and operate without a
- local disk, using filesystems mounted from an NFS server. No system
- modification is necessary, beyond standard configuration files.
- Such a system is easy to set up because all the necessary elements
- are readily available:</para>
+ local disk, using filesystems mounted from an NFS server. No
+ system modification is necessary, beyond standard configuration
+ files. Such a system is relatively easy to set up because all
+ the necessary elements are readily available:</para>
<itemizedlist>
<listitem>
<para>There are at least two possible methods to load the kernel over
@@ -2324,10 +2324,10 @@
<para>There are many ways to set up diskless workstations. Many
elements are involved, and most can be customized to suit local
- taste. The following will describe the setup of a complete system,
- emphasizing simplicity and compatibility with the
- standard FreeBSD startup scripts. The system described has the
- following characteristics:</para>
+ taste. The following will describe variations on the setup of a
+ complete system, emphasizing simplicity and compatibility with
+ the standard FreeBSD startup scripts. The system described has
+ the following characteristics:</para>
<itemizedlist>
<listitem>
@@ -2339,12 +2339,14 @@
configuration files overridden by ones specific to diskless
operation or, possibly, to the workstation they belong to.</para>
<para>The parts of the <filename>root</filename> which have to be
- writable are overlaid with &man.mfs.8; filesystems. Any changes
- will be lost when the system reboots.</para>
+ writable are overlaid with &man.mfs.8; (FreeBSD 4.X) or
+ &man.md.4; (FreeBSD 5.X) filesystems. Any changes will be lost
+ when the system reboots.</para>
</listitem>
<listitem>
- <para>The kernel is loaded by <application>etherboot
- </application>, using DHCP (or BOOTP) and TFTP.</para>
+ <para>The kernel is transferred and loaded either with
+ <application>etherboot</application>, or <emphasis>PXE</emphasis>,
+ as some situations may mandate the use of either method.</para>
</listitem>
</itemizedlist>
@@ -2353,6 +2355,111 @@
other hosts.</para>
</caution>
+ <para>All the information in this section has been tested
+ using FreeBSD releases 4.9-RELEASE-p2 and 5.2-RELEASE. The text is
+ primarily structured for 4.X usage. Notes have been inserted where
+ appropriate to indicate 5.X changes.</para>
+
+
+
+ <sect2>
+ <title>Background information</title>
+
+ <para>Setting up diskless workstations is both relatively
+ straightforward and prone to errors. These are sometimes
+ difficult to diagnose for a number of reasons. For example:
+ <itemizedlist>
+ <listitem>
+ <para>Compile time options may determine the possible values
+ or syntax of runtime parameters.</para>
+ </listitem>
+ <listitem>
+ <para>Error messages are often cryptic or totally
+ absent.</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>In this context, having some knowledge of the background
+ mechanisms involved is very useful to solve the problems that
+ may arise.</para>
+
+ <para>Several operations need to be performed for a successful
+ bootstrap:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>The machine needs to obtain initial
+ parameters such as its IP address, executable file name,
+ server name, root path. This is done using the DHCP or
+ BOOTP protocols. DHCP is a compatible extension of BOOTP,
+ and uses the same port numbers and basic packet
+ format.</para>
+ <para>It is possible to configure a system to use only BOOTP.
+ The &man.bootpd.8; server program is included in the
+ base FreeBSD system.</para>
+
+ <para>However, DHCP has a number of advantages over BOOTP (nicer
+ configuration files, possibility of using PXE, plus many
+ others not directly related to diskless operation), and
+ we shall describe mainly a DHCP configuration, with
+ equivalent exemples using &man.bootpd.8; when possible.
+ The sample configuration will use the ISC DHCP software
+ package (release 3.0.1.r11 was installed on the test
+ server: ).</para>
+ </listitem>
+
+ <listitem>
+ <para>The machine needs to transfer one or several
+ programs to local memory. Either TFTP or NFS are used. The
+ choice between TFTP and NFS is a compile time option in
+ several places. A common source of error is to specify file
+ names for the wrong protocol: TFTP typically transfers all
+ files from a single directory on the server, and would
+ expect simple file names relative to this directory. NFS
+ needs absolute file paths.</para>
+ </listitem>
+
+ <listitem>
+ <para>The possible intermediate bootstrap programs and the
+ kernel need to be initialized and executed. There are
+ several important variations in this area:</para>
+ <itemizedlist>
+ <listitem>
+ <para>When using PXE, the machine will load &man.pxeboot.8;,
+ which is a modified version of the FreeBSD third stage
+ loader. The loader will obtain most parameters necessary
+ to system startup, and leave them in the kernel
+ environment before transferring control. It is possible to
+ use a GENERIC kernel in this case.</para>
+ </listitem>
+
+ <listitem>
+ <para><application>etherboot</application>, will directly
+ load the kernel, with less preparation. A kernel with
+ specific options will need to be built.</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>PXE and <application>etherboot</application> work
+ equally well with 4.X systems. Because 5.X kernels
+ normally let the loader do more work for them, PXE is
+ preferred for 5.X systems.</para>
+ <para>If your BIOS and network
+ cards support PXE, you should probably use it. However,
+ it is still possible to start a 5.2 system with
+ <application>etherboot</application></para>
+
+ </listitem>
+
+ <listitem>
+ <para>Finally, the machine needs to access its filesystems. NFS
+ is used in all cases.</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>See also &man.diskless.8;</para>
+
+ </sect2>
<sect2>
<title>Setup Instructions</title>
@@ -2364,31 +2471,7 @@
<secondary>booting</secondary>
</indexterm>
- <para>There are two protocols that are commonly used to boot a
- workstation that retrieves its configuration over the network: BOOTP
- and DHCP. They are used at several points in the workstation
- bootstrap:</para>
- <itemizedlist>
- <listitem><para><application>etherboot</application> uses
- DHCP (by default) or BOOTP (needs a configuration option) to
- find the kernel. (PXE uses DHCP).</para>
- </listitem>
- <listitem><para>The kernel uses BOOTP to locate the NFS
- root.</para>
- </listitem>
- </itemizedlist>
-
- <para>It is possible to configure a system to use only BOOTP.
- The &man.bootpd.8; server program is included in the
- base FreeBSD system.</para>
-
- <para>However, DHCP has a number of advantages over BOOTP (nicer
- configuration files, possibility of using PXE, plus many others
- not directly related to diskless operation), and we shall describe
- both a pure BOOTP, and a BOOTP+DHCP configuration, with an
- emphasis on the latter, which will use the ISC DHCP software
- package.</para>
-
+
<sect4>
<title>Configuration Using ISC DHCP</title>
<indexterm>
@@ -2399,7 +2482,7 @@
<para>The <application>isc-dhcp</application> server can answer
both BOOTP and DHCP requests.</para>
- <para>As of release 4.4, <application>isc-dhcp
+ <para>As of release 4.9, <application>isc-dhcp
3.0</application> is not part of the base
system. You will first need to install the
<filename role="package">net/isc-dhcp3-server</filename> port or the
@@ -2409,7 +2492,9 @@
<para>Once <application>isc-dhcp</application> is installed, it
needs a configuration file to run, (normally named
<filename>/usr/local/etc/dhcpd.conf</filename>). Here follows
- a commented example:</para>
+ a commented example, where host <literal>margaux</literal>
+ uses <application>etherboot</application> and host
+ <literal>corbieres</literal> uses PXE.</para>
<programlisting>
default-lease-time 600;
@@ -2429,9 +2514,16 @@
hardware ethernet 01:23:45:67:89:ab;
fixed-address margaux.example.com;
next-server 192.168.4.4;<co id="co-dhcp-next-server">
- filename "/tftpboot/kernel.diskless";<co id="co-dhcp-filename">
+ filename "/data/misc/kernel.diskless";<co id="co-dhcp-filename">
option root-path "192.168.4.4:/data/misc/diskless";<co id="co-dhcp-root-path">
}
+ host corbieres {
+ hardware ethernet 00:02:b3:27:62:df;
+ fixed-address corbieres.example.com;
+ filename "pxeboot";<co id="co-dhcp-filenamePxe">
+ next-server 192.168.4.4;
+ option root-path "192.168.4.4:/data/misc/diskless";
+ }
}
</programlisting>
@@ -2446,21 +2538,25 @@
</callout>
<callout arearefs="co-dhcp-next-server"><para>The
- <literal>next-server</literal> directive designates
- the TFTP server (the default is to use the same host as the
- DHCP server).</para>
+ <literal>next-server</literal> directive designates the TFTP
+ or NFS server to use for loading loader or kernel file (the
+ default is to use the same host as the DHCP server).</para>
</callout>
- <callout arearefs="co-dhcp-filename"><para>The
- <literal>filename</literal> directive defines the file that
- <application>etherboot</application> will load as a
- kernel.
- <note><para>PXE appears to prefer a relative file
- name, and it loads <command>pxeboot</command>, not the
- kernel (<literal>option filename
- "pxeboot"</literal>).</para>
- </note>
- </para>
+ <callout arearefs="co-dhcp-filename"><para>The
+ <literal>filename</literal> directive defines the file that
+ <application>etherboot</application> or PXE will load for the
+ next execution step. It must be specified according to the
+ transfer method used. <application>etherboot</application> can
+ be compiled to use NFS or TFTP. The FreeBSD port
+ configures NFS by default.</para>
+ </callout>
+
+ <callout arearefs="co-dhcp-filenamePxe"><para>PXE uses TFTP,
+ which is why a relative file name is used here (this may
+ depend on the TFTP
+ server configuration, but would be fairly typical). Also, PXE
+ loads <filename>pxeboot</filename>, not the kernel.</para>
</callout>
<callout arearefs="co-dhcp-root-path"><para>The
@@ -2478,7 +2574,7 @@
</indexterm>
<para>Here follows an equivalent <command>bootpd</command>
- configuration. This would be found in
+ configuration (reduced to one client). This would be found in
<filename>/etc/bootptab</filename>.</para>
<para>Please note that <application>etherboot</application>
@@ -2529,6 +2625,10 @@
of everything. Else refer to <xref linkend="ports"> for
information about ports and packages.</para>
+ <para>You can change the <application>etherboot</application>
+ configuration (ie: to use TFTP instead of NFS) by editing the
+ Config file in the etherboot source directory.</para>
+
<para>For our setup, we shall use a boot floppy. For other methods
(PROM, or dos program), please refer to the
<application>etherboot</application> documentation.</para>
@@ -2550,6 +2650,22 @@
</sect3>
+ <sect3>
+ <title>Booting with PXE</title>
+
+ <para>By default, the &man.pxeboot.8; loader loads the kernel via NFS.
+ It can be compiled to use TFTP instead by specifying the
+ LOADER_TFTP_SUPPORT option in
+ <filename>/etc/make.conf</filename>. See the comments in
+ <filename>/etc/defaults/make.conf</filename> for
+ instructions.</para>
+
+ <para>To use PXE when the machine starts, you will usually
+ need to select the <literal>Boot from network</literal> option
+ in your BIOS setup, or type a function key during the PC
+ initialization.</para>
+
+ </sect3>
<sect3>
<title>Configuring the TFTP and NFS Servers</title>
@@ -2563,8 +2679,10 @@
<secondary>diskless operation</secondary>
</indexterm>
- <para>You need to enable <command>tftpd</command> on the TFTP
- server:</para>
+ <para>If any of the boostrap programs (PXE, pxeboot, or etherboot)
+ are configured to use TFTP, you need to enable
+ <command>tftpd</command> on the file server:</para>
+
<procedure>
<step>
<para>Create a directory from which <command>tftpd</command>
@@ -2575,7 +2693,7 @@
<para>Add this line to your
<filename>/etc/inetd.conf</filename>:</para>
- <programlisting>tftp dgram udp wait root /usr/libexec/tftpd tftpd -s /tftpboot</programlisting>
+ <programlisting>tftp dgram udp wait root /usr/libexec/tftpd tftpd -s /tftpboot</programlisting>
<note><para>It appears that at least some PXE versions want
the TCP version of TFTP. In this case, add a second line,
@@ -2595,7 +2713,7 @@
location is set in both <filename>inetd.conf</filename> and
<filename>dhcpd.conf</filename>.</para>
- <para>You also need to enable NFS and export the
+ <para>In all cases, you also need to enable NFS and export the
appropriate filesystem on the NFS server.</para>
<procedure>
@@ -2611,7 +2729,7 @@
point and replace <replaceable>margaux</replaceable>
with the name of the diskless workstation):</para>
- <programlisting><replaceable>/data/misc</replaceable> -alldirs -ro <replaceable>margaux</replaceable></programlisting>
+ <programlisting><replaceable>/data/misc</replaceable> -alldirs -ro <replaceable>margaux corbieres</replaceable></programlisting>
</step>
<step>
<para>Tell <command>mountd</command> to reread its configuration
@@ -2632,23 +2750,48 @@
<secondary>kernel configuration</secondary>
</indexterm>
- <para>Create a kernel configuration file for the diskless client
- with the following options (in addition to the usual
- ones):</para>
+ <para>If using <application>etherboot</application>, you need to
+ create a kernel configuration file for the diskless client
+ with the following options (in addition to the usual ones):</para>
<programlisting>
- options BOOTP # Use BOOTP to obtain IP address/hostname
- options BOOTP_NFSROOT # NFS mount root filesystem using BOOTP info
- options BOOTP_COMPAT # Workaround for broken bootp daemons.
+ options BOOTP # Use BOOTP to obtain IP address/hostname
+ options BOOTP_NFSROOT # NFS mount root filesystem using BOOTP info
</programlisting>
- <para>You may also want to use <literal>BOOTP_NFSV3</literal> and
- <literal>BOOTP_WIRED_TO</literal> (refer to <filename>LINT</filename>).</para>
-
+ <para>You may also want to use <literal>BOOTP_NFSV3</literal>,
+ <literal>BOOT_COMPAT</literal> and
+ <literal>BOOTP_WIRED_TO</literal> (refer to
+ <filename>LINT</filename>).</para>
+
+ <para>These option names are historical and slightly
+ misleading as they actually enable indifferent use of DHCP or BOOTP
+ inside the kernel (it is also possible to force strict BOOTP or
+ DHCP use).</para>
+
<para>Build the kernel (See <xref linkend="kernelconfig">),
- and copy it to the tftp directory, under the name listed
- in <filename>dhcpd.conf</filename>.</para>
+ and copy it to the place specified in
+ <filename>dhcpd.conf</filename>.</para>
+ <note><para>When using PXE, building a kernel with the above
+ options is not strictly necessary. Enabling them will
+ cause more DHCP requests to be issued during kernel
+ startup, with a small risk of inconsistency between the
+ new values and those retrieved by pxeboot in some special
+ cases. The advantage of using them is that the host name
+ will be set as a side effect. Else you will need to set
+ the host name by another method, for example in a
+ client-specific <filename>rc.conf</filename> file.</para>
+ </note>
+
+ <note><para>In order to be loadable with
+ <application>etherboot</application>, a 5.x kernel needs
+ to have the device hints compiled in. You would typically
+ set the following option in the configuration file (see
+ the NOTES configuration comments file).
+ <programlisting>hints "GENERIC.hints" </programlisting>
+ </para>
+ </note>
</sect3>
@@ -2671,6 +2814,13 @@
the place where the filesystem will be created (the
<literal>DEST</literal> variable).</para>
+ <note><para>The <filename>clone_root</filename> script distributed
+ with FreeBSD 5.2 is out of date relative to the 4.x version, and
+ also needs a few specific 5.x adjustments. Please refer to PR
+ <ulink url="http://www.freebsd.org/cgi/query-pr.cgi?pr=misc/62417">
+ misc/62417</ulink>.</para>
+ </note>
+
<para>Refer to the comments at the top of the script for
instructions. They explain how the base filesystem is built,
and how files may be selectively overridden by versions specific
@@ -2689,35 +2839,64 @@
confusing. Use them for reference only, except if you prefer
the method that they describe, in which case you will need
customized <filename>rc</filename> scripts.</para>
+
+ <note><para>When using PXE, it may happen that the loader will
+ get a wrong idea of the root filesystem to be used by the
+ kernel (this will be the case if the cloned root
+ <filename>/etc/fstab</filename> is a copy of the server's,
+ as the client-specific version will not be visible by the
+ loader, which looks for hints there).</para>
+ <para>This will cause
+ error messages at the end of the kernel initialization,
+ when the kernel tries to mount the wrong root and fails,
+ or even actual errors if it succeeds because a local disk
+ happens to be present. </para>
+
+ <para>It is possible to prevent this problem from happening
+ by inserting the following line in
+ <filename>/boot/loader.conf</filename>:</para>
+ <programlisting>
+ vfs.root.mountfrom=""
+ </programlisting>
+ </note>
+
+
</sect3>
<sect3>
<title>Configuring Swap</title>
<para>If needed, a swap file located on the server can be
- accessed via NFS. The exact <filename>bootptab</filename>
- or <filename>dhcpd.conf</filename> options are not clearly
- documented at this time. The following configuration
- suggestions have been reported to work in some installations
- using isc-dhcp 3.0rc11.</para>
+ accessed via NFS. One of the methods commonly used to do this
+ has been discontinued in release 5.X.</para>
+
+ <sect4>
+ <title>NFS swap with FreeBSD 4.x</title>
+
+ <para>The swap file location and size can be specified with
+ BOOTP/DHCP FreeBSD-specific options 128 and 129. Examples of
+ configuration files for isc-dhcp 3 or bootpd follow.</para>
+
<procedure>
<step><para>Add the following lines to
<filename>dhcpd.conf</filename>:</para>
<programlisting>
- # Global section
- option swap-path code 128 = string;
- option swap-size code 129 = integer 32;
-
- host margaux {
- ... # Standard lines, see above
- option swap-path <replaceable>"192.168.4.4:/netswapvolume/netswap"</replaceable>;
- option swap-size <replaceable>64000</replaceable>;
- }
+ # Global section
+ option swap-path code 128 = string;
+ option swap-size code 129 = integer 32;
+
+ host margaux {
+ ... # Standard lines, see above
+ option swap-path <replaceable>"192.168.4.4:/netswapvolume/netswap"</replaceable>;
+ option swap-size <replaceable>64000</replaceable>;
+ }
</programlisting>
- <para>The idea is that, at least for a FreeBSD client,
- DHCP/BOOTP option code 128 is the path to the NFS swap file,
- and option code 129 is the swap size in kilobytes. Older
- versions of <command>dhcpd</command> allowed a syntax of
+
+ <para><literal>swap-path</literal> is the path to a directory
+ where swap files will be located. Each file will be named
+ <filename>swap.<replaceable>clientIp</replaceable></filename>.</para>
+
+ <para>Older versions of <command>dhcpd</command> used a syntax of
<literal>option option-128 "...</literal>, which is no
longer supported.</para>
<para><filename>/etc/bootptab</filename> would use the
@@ -2733,12 +2912,12 @@
<step>
<para>On the NFS swap file server, create the swap
file(s)</para>
- <screen>
- &prompt.root; <userinput>mkdir <replaceable>/netswapvolume/netswap</replaceable></userinput>
- &prompt.root; <userinput>cd <replaceable>/netswapvolume/netswap</replaceable></userinput>
- &prompt.root; <userinput>dd if=/dev/zero bs=1024 count=<replaceable>64000</replaceable> of=swap.<replaceable>192.168.4.6</replaceable></userinput>
- &prompt.root; <userinput>chmod 0600 swap.<replaceable>192.168.4.6</replaceable></userinput>
- </screen>
+ <screen>
+ &prompt.root; <userinput>mkdir <replaceable>/netswapvolume/netswap</replaceable></userinput>
+ &prompt.root; <userinput>cd <replaceable>/netswapvolume/netswap</replaceable></userinput>
+ &prompt.root; <userinput>dd if=/dev/zero bs=1024 count=<replaceable>64000</replaceable> of=swap.<replaceable>192.168.4.6</replaceable></userinput>
+ &prompt.root; <userinput>chmod 0600 swap.<replaceable>192.168.4.6</replaceable></userinput>
+ </screen>
<para><replaceable>192.168.4.6</replaceable> is the IP address
for the diskless client.</para>
</step>
@@ -2747,13 +2926,31 @@
<para>On the NFS swap file server, add the following line to
<filename>/etc/exports</filename>:</para>
<programlisting>
- <replaceable>/netswapvolume</replaceable> -maproot=0:10 -alldirs <replaceable>margaux</replaceable>
+<replaceable>/netswapvolume</replaceable> -maproot=0:10 -alldirs <replaceable>margaux</replaceable>
</programlisting>
<para>Then tell <application>mountd</application> to reread the
exports file, as above.</para>
</step>
</procedure>
+ </sect4>
+
+ <sect4>
+ <title>NFS swap with FreeBSD 5.x</title>
+
+ <para>The kernel does not support enabling NFS swap at boot
+ time. Swap must be enabled by the startup scripts, by
+ mounting a writeable file system and creating and enabling a
+ swap file. This can be done, for example, in
+ <filename>/etc/rc.local</filename>. Example:</para>
+
+ <screen>
+ swapfile=/writableVolume/swap/swap.<replaceable>MyIP</replaceable>
+ dd if=/dev/zero of=$swapfile bs=1k count=1 oseek=100000
+ swapon $swapfile
+ rm -rf $swapfile
+ </screen>
+ </sect4>
</sect3>
<sect3>
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-doc
mailing list