docs/54082: Update to Mirror Site Docs
Ken Smith
kensmith at cse.Buffalo.EDU
Fri Jul 4 07:20:20 UTC 2003
>Number: 54082
>Category: docs
>Synopsis: Update to Mirror Site Docs
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: freebsd-doc
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: update
>Submitter-Id: current-users
>Arrival-Date: Fri Jul 04 00:20:15 PDT 2003
>Closed-Date:
>Last-Modified:
>Originator: Ken Smith
>Release: FreeBSD 5.1-RELEASE i386
>Organization:
U. Buffalo CSE Department/FreeBSD.org
>Environment:
System: FreeBSD hobbes.cse.buffalo.edu 5.1-RELEASE FreeBSD 5.1-RELEASE #2: Thu Jun 5 20:46:57 EDT 2003 root at hobbes.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386
>Description:
I was given the position of Mirror Site Coordinator (I'm now one
of mirror-admin at freebsd.org). My first task was to update the
Mirror Site Docs to reflect some new procedures in place.
>How-To-Repeat:
Not Applicable
>Fix:
Apply diffs below to /usr/doc/en_US.ISO8859-1/articles/hubs/article.sgml
please. Thanks.
--- article.sgml.orig Fri May 30 23:18:28 2003
+++ article.sgml Fri Jul 4 03:05:53 2003
@@ -26,7 +26,7 @@
<firstname>Valentino</firstname>
<surname>Vaschetto</surname>
<affiliation>
- <address><email>logo at FreeBSD.org</email></address>
+ <address><email>logo at FreeBSD.org</email></address>
</affiliation>
</author>
<author>
@@ -36,13 +36,28 @@
<address><email>dl at leo.org</email></address>
</affiliation>
</author>
+ <author>
+ <firstname>Ken</firstname>
+ <surname>Smith</surname>
+ <affiliation>
+ <address><email>kensmith at cse.buffalo.edu</email></address>
+ </affiliation>
+ </author>
</authorgroup>
<abstract>
<para>An in-progress article on how to mirror FreeBSD, aimed at
- hub administrators</para>
+ hub administrators.</para>
</abstract>
</articleinfo>
+ <sect1 id="mirror-contact">
+ <title>Contact Information</title>
+ <para>
+ The Mirror System Coordinators can be reached through email
+ at <email>mirror-admin at FreeBSD.org</email>. There is also
+ a &a.hubs;.
+ </para>
+ </sect1>
<sect1 id="mirror-requirements">
<title>Requirements for FreeBSD mirrors</title>
<sect2 id="mirror-diskspace">
@@ -50,22 +65,22 @@
<para>
Disk space is one of the most important requirements.
Depending on the set of releases, architectures,
- and degree of completeness you want to mirror, a huge
- amount of disk space may be consumed. Also keep in mind,
+ and degree of completeness you want to mirror a huge
+ amount of disk space may be consumed. Also keep in mind
that <emphasis>official</emphasis> mirrors are probably required to be
complete. The CVS repository and the web pages should
- always be mirrored completely. Also note, that the
- numbers stated here, are reflecting the current
- state (at 4.5-RELEASE). Further development and
+ always be mirrored completely. Also note that the
+ numbers stated here are reflecting the current
+ state (at 4.8-RELEASE/5.1-RELEASE). Further development and
releases will only increase the required amount.
- Also make sure, to keep some (ca. 10-20%) extra space
- around, just to be sure.
+ Also make sure to keep some (ca. 10-20%) extra space
+ around just to be sure.
Here are some approximate figures:
</para>
<itemizedlist>
- <listitem><para>Full FTP Distribution: 100 GB</para></listitem>
- <listitem><para>CVS repository: 2 GB</para></listitem>
- <listitem><para>CTM deltas: 1.5 GB</para></listitem>
+ <listitem><para>Full FTP Distribution: 126 GB</para></listitem>
+ <listitem><para>CVS repository: 2.7 GB</para></listitem>
+ <listitem><para>CTM deltas: 1.8 GB</para></listitem>
<listitem><para>Webpages: 300 MB</para></listitem>
</itemizedlist>
</sect2>
@@ -75,28 +90,29 @@
Of course, you need to be connected to the Internet.
The required bandwidth depends on your intended use
of the mirror. If you just want to mirror some
- parts of FreeBSD for local use at your site/intranet,
- the demand may be much smaller, than if you want to
- make the files publicly available or even if you intend
- to become an official mirror. We can only give rough
+ parts of FreeBSD for local use at your site/intranet
+ the demand may be much smaller than if you want to
+ make the files publicly available. If you intend
+ to become an official mirror the bandwidth required will
+ be even higher. We can only give rough
estimates here:
</para>
<itemizedlist>
<listitem><para>Local site, no public access: basically no minimum,
- but < 2 Mbps could make syncing a pain.</para></listitem>
+ but < 2 Mbps could make syncing too slow.</para></listitem>
<listitem><para>Unofficial public site: 34 Mbps is probably a good start.</para></listitem>
- <listitem><para>Official site: > 100 Mbps is recommended, also your host
+ <listitem><para>Official site: > 100 Mbps is recommended, and your host
should be connected as close as possible to your border router.</para></listitem>
</itemizedlist>
</sect2>
<sect2 id="mirror-system">
<title>System Requirements, CPU, RAM</title>
<para>
- This also depends on the expected amount of clients,
- which is determined by the servers policy. It is
+ One thing this depends on the expected number of clients,
+ which is determined by the server's policy. It is
also affected by the types of services you want to offer.
Plain FTP or HTTP services may not require a huge
- amount of resources. Watch out, if you provide
+ amount of resources. Watch out if you provide
CVSup, rsync or even AnonCVS. This can have a huge
impact on CPU and memory requirements. Especially
rsync is considered a memory hog, and CVSup does
@@ -107,26 +123,26 @@
are just examples to give you a very rough hint.
</para>
<para>
- For a moderately visited site, that offers
- <application>rsync</application>, you might
+ For a moderately visited site that offers
+ <application>rsync</application> you might
consider a current CPU with around 800MHz - 1 GHz,
and at least 512MB RAM. This is probably the
minimum you want for an <emphasis>official</emphasis>
site.
</para>
<para>
- For a frequently used site you need definitely
- more RAM (consider 2GB as a good start),
- and possibly more CPU, which could also mean,
+ For a frequently used site you definitely need
+ more RAM (consider 2GB as a good start)
+ and possibly more CPU, which could mean
that you need to go for a SMP system.
</para>
<para>
You also want to consider a fast disk subsystem.
Operations on the CVS repository require a fast
- disk subsystem (RAID is greatly advised). A SCSI
+ disk subsystem (RAID is highly advised). A SCSI
controller that has a cache of its own can also
- speed up things, since most of these services incur a
- very large number of small modifications to the disk.
+ speed up things since most of these services incur a
+ large number of small modifications to the disk.
</para>
<para>You can also experiment with enlarging the portion
of system memory which is used for the file system buffer cache.
@@ -139,8 +155,8 @@
<title>Services to offer</title>
<para>
Every mirror site is required to have a set of core services
- available. In addition to these basic services, which mirrors are
- required to provide, there is a number of optional services that
+ available. In addition to these required services there are a
+ number of optional services that
server administrators may choose to offer. This section explains
which services you can provide and how to go about implementing them.
</para>
@@ -148,7 +164,7 @@
<title>FTP (required for FTP fileset)</title>
<para>
This is one of the most basic services, and
- it is required for each mirror, offering public
+ it is required for each mirror offering public
FTP distributions. FTP access must be
anonymous, and no upload/download ratios
are allowed (a ridiculous thing anyway).
@@ -194,21 +210,26 @@
FreeBSD's <application>ftpd</application>, <application>proftpd</application>,
<application>wu-ftpd</application> and maybe <application>ncftpd</application>
are among the most commonly ones.
- The others do not have a large userbase among mirror sites.
+ The others do not have a large userbase among mirror sites. One
+ thing to consider is that you may need flexibility in limiting
+ how many simultaneous connections are allowed, thus limiting how
+ much network bandwidth and system resources are consumed.
</para>
</sect3>
<sect3 id="mirror-serv-rsync">
<title>RSYNC (optional for FTP fileset)</title>
<para>
- <application>Rsync</application> is often also offered for convenience, for the
- contents of the FTP area of FreeBSD. The
- protocol is different from FTP in many ways, and
- overall, it can be stated, that it is much more
+ <application>Rsync</application> is often offered for access to the
+ contents of the FTP area of FreeBSD so other Mirror Sites can use
+ your system as their source. The
+ protocol is different from FTP in many ways.
+ It is much more
bandwidth friendly, as only differences between files
- are transferred, not whole files.
- <application>Rsync</application> does require significant amount of memory for
+ are transferred instead of whole files when they change.
+ <application>Rsync</application> does require a significant
+ amount of memory for
each instance. The size depends on the size of
- the synced module in terms of number of directories and
+ the synced module in terms of the number of directories and
files. <application>Rsync</application> can use <command>rsh</command> and
<command>ssh</command> (now default) as a transport,
or use it's own protocol for stand-alone access
@@ -242,7 +263,7 @@
<listitem>
<para><filename role="package">www/thttpd</filename>:
- If you are going to be serving a lot amount of static content
+ If you are going to be serving a large amount of static content
you may find that using an application such as tHttpd is more
efficient than Apache. It is optimized for excellent performance
on FreeBSD.</para>
@@ -263,10 +284,10 @@
<title>CVSup (desired for CVS repository)</title>
<para>
<application>CVSup</application> is a very efficient way of distributing files.
- It works similar as rsync, but was specially designed for
- the use with CVS repositories. If you want to offer the
- FreeBSD CVS repository, you really want to consider
- offering it via <application>CVSup</application>. Still it is possible to offer
+ It works similar to rsync, but was specially designed for
+ use with CVS repositories. If you want to offer the
+ FreeBSD CVS repository you really should consider
+ offering it via <application>CVSup</application>. It is possible to offer
the CVS repository via <application>AnonCVS</application>, FTP,
<application>Rsync</application> or HTTP, but
people would benefit much more from <application>CVSup</application> access.
@@ -274,7 +295,7 @@
It is a bit tricky to install on non-FreeBSD platforms,
since it is written in Modula-3 and therefore requires
a Modula-3 environment. John Polstra has built a
- stripped down version of M3, that is sufficient to
+ stripped down version of M3 that is sufficient to
run <application>CVSup</application>, and can be installed much easier.
See <ulink url="http://www.polstra.com/projects/freeware/ezm3/">Ezm3</ulink>
for details. Related ports are:
@@ -286,8 +307,8 @@
</listitem>
<listitem>
<para><filename role="package">net/cvsup-mirror</filename>: The CVSup mirror kit, which requires
- <filename role="package">net/cvsup</filename>, and configures it mirror-ready. Some
- site administrators may want a different setup, though.
+ <filename role="package">net/cvsup</filename>,and configures it mirror-ready. Some
+ site administrators may want a different setup though.
</para>
</listitem>
</itemizedlist>
@@ -296,30 +317,30 @@
<filename role="package">net/cvsup-without-gui</filename> you might want to have
a look at. If you prefer a static binary package, take a look
<ulink url="http://people.FreeBSD.org/~jdp/s1g/">here</ulink>.
- This page still refers to the S1G bug, that was present
+ This page still refers to the S1G bug that was present
in <application>CVSup</application>. Maybe
John will setup a generic download-site to get
static binaries for various platforms.
</para>
<para>
It is possible to use CVSup to offer
- any kind of fileset, not just CVS repositories,
- but configuration can be complex.
- CVSup is known to eat some CPU on the server as on the
+ any kind of fileset, not just CVS repositories.
+ But configuration can be complex.
+ CVSup is known to eat some CPU on both the server and the
client, since it needs to compare lots of files.
</para>
</sect3>
<sect3 id="mirror-anoncvs">
<title>AnonCVS (optional for CVS repository)</title>
<para>
- If you have the CVS repository, you may want to offer
+ If you have the CVS repository you may want to offer
anonymous CVS access. A short warning first:
- There is not that much demand for it,
- and it requires some experience and you need to know,
+ There is not much demand for it,
+ it requires some experience, and you need to know
what you are doing.
</para>
<para>
- Generally there are two ways, how
+ Generally there are two ways
to access a CVS repository remotely: via
<emphasis>pserver</emphasis> or via <command>ssh</command>
(we don't consider <command>rsh</command>).
@@ -330,7 +351,7 @@
in the CVS repository, to be used as a login-shell for the
anonymous ssh account. It does a chroot, and therefore
requires the CVS repository to be available under the
- anonymous user's home-directory, which may not be possible
+ anonymous user's home directory. This may not be possible
for all sites. If you just offer <emphasis>pserver</emphasis>
this restriction does not apply, but you may run with
more security risks. You don't need to install any special
@@ -341,19 +362,19 @@
<programlisting>
cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --allow-root=/home/ncvs pserver
</programlisting>
- See the manpage for details of the options. See also the cvs <emphasis>info</emphasis>
- page, about additional ways to make sure, access is read-only.
- It is advisable, that you create an unprivileged account,
+ See the manpage for details of the options. Also see the cvs <emphasis>info</emphasis>
+ page about additional ways to make sure access is read-only.
+ It is advised that you create an unprivileged account,
preferably called <username>anoncvs</username>.
Also you need to create a file <filename>passwd</filename>
in your <filename>/home/ncvs/CVSROOT</filename> and assign a
CVS password (empty or <literal>anoncvs</literal>) to that user.
The directory <filename>/anoncvstmp</filename> is a special
purpose memory based file system. It is not required but
- advised, since &man.cvs.1; creates a shadow directory
+ advised since &man.cvs.1; creates a shadow directory
structure in your <filename>/tmp</filename> which is
- not used after the operation, but slows things
- dramatically, if real disk operations are required.
+ not used after the operation but slows things
+ dramatically if real disk operations are required.
Here is an excerpt from <filename>/etc/fstab</filename>,
how to set up such a MFS:
<programlisting>
@@ -365,9 +386,9 @@
</sect2>
</sect1>
<sect1 id="mirror-howto">
- <title>How to mirror FreeBSD</title>
+ <title>How To Mirror FreeBSD</title>
<para>
- Ok now, you know the requirements, and how to offer
+ Ok, now you know the requirements and how to offer
the services, but not how to get it. :-)
This section explains how to actually mirror
the various parts of FreeBSD, what tools to use,
@@ -376,23 +397,22 @@
<sect2 id="mirror-ftp">
<title>FTP</title>
<para>
- The FTP area is the largest amount of data, that
- needs to be mirrored. It includes the <emphasis>distributions
- sets</emphasis>, required for network installation, the
- <emphasis>branches</emphasis>, that are actually snapshots
+ The FTP area is the largest amount of data that
+ needs to be mirrored. It includes the <emphasis>distribution
+ sets</emphasis> required for network installation, the
+ <emphasis>branches</emphasis> which are actually snapshots
of checked-out source trees, the <emphasis>ISO Images</emphasis>
to write CD-ROMs with the installation distribution,
- a live file system, and lots of packages, the ports tree,
- distfiles and a huge amount of packages. All of course
- for various FreeBSD versions, and <emphasis>i386</emphasis>
- and <emphasis>alpha</emphasis> architecture.
+ a live file system, lots of packages, the ports tree,
+ distfiles, and a huge amount of packages. All of course
+ for various FreeBSD versions, and various architectures.
</para>
<sect3 id="mirror-ftp-ftp">
<title>With FTP mirror</title>
<para>
You can use a <application>FTP mirror</application>
- program, to get the files. There are a lot around, and
- widely used, like:
+ program to get the files. There are a lot around and
+ widely used, for example:
<itemizedlist>
<listitem><para><filename role="package">ftp/mirror</filename></para></listitem>
<listitem><para><filename role="package">ftp/ftpmirror</filename></para></listitem>
@@ -404,40 +424,38 @@
<filename role="package">ftp/mirror</filename> was very popular, but seemed
to have some drawbacks, as it is written in &man.perl.1;,
- and did have real problems on mirroring large
- directories like a FreeBSD site. There are rumors, that
- the current version has fixed this, by allowing
- to specify a different algorithm for comparing
- the directory structure.
+ and had real problems with mirroring large
+ directories like a FreeBSD site. There are rumors that
+ the current version has fixed this by allowing
+ a different algorithm for comparing
+ the directory structure to be specified.
</para>
<para>
- In general FTP is not really good for mirroring, since it transfers
- each whole file, if it has changed, and does
- not create a single data stream, that will benefit from
+ In general FTP is not really good for mirroring. It transfers
+ the whole file if it has changed, and does
+ not create a single data stream which would benefit from
a large TCP congestion window.
</para>
</sect3>
<sect3 id="mirror-ftp-rsync">
<title>With RSYNC</title>
<para>
- A better way, to mirror the FTP area is <application>rsync</application>.
+ A better way to mirror the FTP area is <application>rsync</application>.
You can install the port <filename role="package">net/rsync</filename> and then use
rsync to sync with your upstream host.
<application>rsync</application> is already mentioned
in <xref linkend="mirror-serv-rsync">.
Since <application>rsync</application> access is not
- required, your preferred upstream site may not allow it.
- Since it is quite common, though, chances are small, that
- you cannot use it. You can always consider using an
- upstream server, that offers it, just for the benefits
- of rsync.
+ required your preferred upstream site may not allow it.
+ You may need to hunt around a little bit to find a site
+ that allows <application>rsync</application> access.
<note>
<para>
Since the number of <application>rsync</application>
clients will have a significant impact on the server
machine, most admins impose limitations on their
- server. For a mirror, you should ask the site maintainer
- you are syncing from, about their policy, and maybe
+ server. For a mirror you should ask the site maintainer
+ you are syncing from about their policy, and maybe make
an exception for your host (since you are a mirror).
</para>
</note>
@@ -452,7 +470,7 @@
be aware that the module-directory (here "FreeBSD")
will not be created, so you cannot omit the target directory.
Also you might
- want to set up a script framework, that calls such a command
+ want to set up a script framework that calls such a command
via &man.cron.8;.
</para>
</sect3>
@@ -462,9 +480,9 @@
A few sites, including the one-and-only <hostid>ftp-master.FreeBSD.org</hostid>
even offer <application>CVSup</application> to mirror the contents of
the FTP space. You need to install a <application>cvsup</application>
- client, preferably from the port: <filename role="package">net/cvsup</filename>.
+ client, preferably from the port <filename role="package">net/cvsup</filename>.
(Also reread <xref linkend="mirror-serv-cvsup">.)
- A sample supfile, suitable for <hostid>ftp-master.FreeBSD.org</hostid>
+ A sample supfile suitable for <hostid>ftp-master.FreeBSD.org</hostid>
looks like this:
<programlisting>
#
@@ -484,9 +502,8 @@
</programlisting>
It seems <application>CVSup</application> would be the best
- way to mirror the archive, in terms of efficiency, but
- it is only available from few sites. In fact I just know
- <hostid>ftp-master.FreeBSD.org</hostid> for sure.
+ way to mirror the archive in terms of efficiency, but
+ it is only available from a few sites.
<note id="mirror-cvsup-s-option">
<para>
Please have look at the <application>CVSup</application> documentation
@@ -501,8 +518,8 @@
<sect2 id="mirror-cvs">
<title>Mirroring the CVS repository</title>
<para>
- Again you have various possibilities, but the most
- recommended one, is to use <link linkend="mirror-cvs-cvsup">CVSup</link>.
+ Again you have various possibilities but the most
+ recommended one is to use <link linkend="mirror-cvs-cvsup">CVSup</link>.
</para>
<sect3 id="mirror-cvs-cvsup">
<title>Using CVSup</title>
@@ -533,7 +550,7 @@
</para>
<note>
<para>
- Please don't forget to consider the hint,
+ Please don't forget to consider the hint
mentioned in <link linkend="mirror-cvsup-s-option">this note</link>
above.
</para>
@@ -554,16 +571,16 @@
<listitem><para>maybe even <application>HTTP</application></para></listitem>
</itemizedlist>
- If you find a site, that supports it, you could use
- <filename role="package">net/sup</filename>, but it is inferior to <application>CVSup</application>
+ If you find a site that supports it you could use
+ <filename role="package">net/sup</filename>. But it is inferior to <application>CVSup</application>
and it's deficiencies caused John Polstra to develop
- <application>CVSup</application> in the first place, so
+ <application>CVSup</application> in the first place so
it is clearly not recommended.
<important>
<para>
You can <emphasis>NOT</emphasis> use AnonCVS to
- mirror the CVS repository, since CVS does not allow
+ mirror the CVS repository since CVS does not allow
you to access the repository itself, but only checked
out versions of the modules.
</para>
@@ -574,13 +591,13 @@
<sect2 id="mirror-www">
<title>Mirroring the WWW pages</title>
<para>
- The best way is, to check out the <emphasis>www</emphasis>
+ The best way is to check out the <emphasis>www</emphasis>
distribution from CVS. If you have a local mirror of the
- CVS repository, it is probably as easy as:
+ CVS repository it is probably as easy as:
<screen>&prompt.user; <userinput>cvs -d /home/ncvs co www</userinput></screen>
and a <emphasis>cronjob</emphasis>, that calls <command>cvs up -d -P</command>
on a regular basis, maybe just after your repository was updated.
- Of course, the files need to remain in a directory, available
+ Of course, the files need to remain in a directory available
for public WWW access. The installation and configuration of a
webserver is not discussed here.
</para>
@@ -588,7 +605,7 @@
<note><para>For the website to be visible, users must execute the &man.make.1;
command in the main <filename>www</filename> directory. This command
will create the standard <filename>*.html</filename> files for web
- viewing. For this to work, however, the
+ viewing. For this to work however, the
<filename role="package">textproc/docproj</filename> port must be
installed.</para></note>
<para>
@@ -615,16 +632,16 @@
</programlisting>
</para>
<para>
- Using <filename role="package">ftp/wget</filename> or other web-mirror tools, is
+ Using <filename role="package">ftp/wget</filename> or other web-mirror tools is
probably not recommended.
</para>
<sect3 id="mirror-www-doc">
<title>Mirroring the FreeBSD documentation</title>
<para>
- As the documentation is referenced a lot from the
- webpages, it is recommended, that you mirror the
+ Since the documentation is referenced a lot from the
+ webpages it is recommended that you mirror the
FreeBSD documentation as well. However, this is not
- so trivial as the www-pages alone.
+ as trivial as the www-pages alone.
</para>
<para>
First of all, you should get the doc sources,
@@ -650,27 +667,27 @@
</para>
<para>
Then you need to install a couple of ports.
- You are lucky, that there is a meta-port:
+ You are lucky, there is a meta-port:
<filename role="package">textproc/docproj</filename> to do the work
- for you. You need to setup some
+ for you. You need to set up some
environment variables, like
- <literal>SGML_CATALOG_FILES</literal>,
- also have a look at your <filename>/etc/make.conf</filename>
+ <literal>SGML_CATALOG_FILES</literal>.
+ Also have a look at your <filename>/etc/make.conf</filename>
(copy <filename>/etc/defaults/make.conf</filename> if
- you don't have one), and look at the
+ you don't have one) and look at the
<literal>DOC_LANG</literal> variable.
Now you are probably ready to run <command>make</command>
in you doc directory (<filename>/usr/share/doc</filename>
by default) and build the documentation.
Again you need to make it accessible for your webserver
- and make sure, the links point to the right location.
+ and make sure the links point to the right location.
<important>
<para>
The building of the documentation, as well as lots
of side issues, is documented itself in:
<ulink url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/fdp-primer/">fdp-primer</ulink>.
Please read this piece of documentation, especially if you
- have problems, building the documentation.
+ have problems building the documentation.
</para>
</important>
<note>
@@ -686,7 +703,7 @@
<para>
Every mirror should be updated on a regular
basis. You will certainly need some script
- framework for it, that will be called by
+ framework for it that will be called by
&man.cron.8;. Since nearly every admin
does this his own way, we cannot give
specific instructions. It could work
@@ -702,19 +719,24 @@
</step>
<step>
<para>
- Add some output redirections, so diagnostic
+ Add some output redirections so diagnostic
messages are logged to a file.
</para>
</step>
<step>
<para>
- Test, if your script works. Check the logs.
+ Test if your script works. Check the logs.
</para>
</step>
<step>
<para>
- Use &man.crontab.1; to add the script the
- appropriate user's &man.crontab.5;.
+ Use &man.crontab.1; to add the script to the
+ appropriate user's &man.crontab.5;. This should be a
+ different user than what your FTP daemon runs as so that
+ if file permissions inside your FTP area are not world-readable
+ those files can not be accessed by anonymous FTP. This is used
+ to "stage" releases - making sure all of the Official Mirror
+ sites have all of the necessary Release files on Release Day.
</para>
</step>
</procedure>
@@ -732,53 +754,61 @@
<title>Where to mirror from</title>
<para>
This is an important issue. So this section will
- spend some effort to explain the backgrounds.
+ spend some effort to explain the backgrounds. We will say this
+ several times : Under no circumstances should you mirror from
+ <hostid>ftp.FreeBSD.org</hostid>.
</para>
<sect2 id="mirror-where-organization">
<title>A few words about the organization</title>
<para>
Mirrors are organized by country. All
official mirrors have a DNS entry of the form
- <hostid>ftpN.CC.FreeBSD.org</hostid>. With
- <emphasis>CC</emphasis> (i.e. country code) being the
- <emphasis>top level domain</emphasis>
- of the country, where this mirror is located;
- and <emphasis>N</emphasis> is a number,
+ <hostid>ftpN.CC.FreeBSD.org</hostid>. <emphasis>CC</emphasis>
+ (i.e. country code) is the
+ <emphasis>top level domain</emphasis> (TLD)
+ of the country where this mirror is located.
+ <emphasis>N</emphasis> is a number,
telling that the host would be the <emphasis>Nth</emphasis>
mirror in that country.
(Same applies to <hostid>cvsupN.CC.FreeBSD.org</hostid>,
<hostid>wwwN.CC.FreeBSD.org</hostid>, etc.)
There are mirrors with no <emphasis>CC</emphasis> part.
- These are usually located in the US, but don't need to.
- <hostid>ftp.FreeBSD.org</hostid> is currently
- located in Denmark and just another mirror
- (i.e. it is <emphasis>NO</emphasis> master site).
+ These are the mirror sites that are very well connected and
+ allow a large number of concurrent users.
+ <hostid>ftp.FreeBSD.org</hostid> is actually two machines, one currently
+ located in Denmark and the other in the United States.
+ It is <emphasis>NOT</emphasis> a master site and should never be
+ used to mirror from. Lots of online documentation leads "interactive"
+ users to <hostid>ftp.FreeBSD.org</hostid> so automated mirroring
+ systems should find a different machine to mirror from.
</para>
<para>
Additionally there exists a hierarchy of mirrors, which
- is described terms of <emphasis>tiers</emphasis>.
- The master sites are not referred to, but can be
+ is described in terms of <emphasis>tiers</emphasis>.
+ The master sites are not referred to but can be
described as <emphasis>Tier-0</emphasis>. Mirrors
that mirror from these sites can be considered
<emphasis>Tier-1</emphasis>, mirrors of <emphasis>Tier-1</emphasis>-mirrors,
are <emphasis>Tier-2</emphasis>, etc.
Official sites are encouraged to be of a low <emphasis>tier</emphasis>,
- but the lower the tier, the higher the requirements in
+ but the lower the tier the higher the requirements in
terms as described in <xref linkend="mirror-requirements">.
- Also access to low-tier-mirrors may be restricted, and
+ Also access to low-tier-mirrors may be restricted and
access to master sites is definitely restricted.
The <emphasis>tier</emphasis>-hierarchy is not reflected
- by DNS and generally not documented anywhere, except
- for the master sites. However, official mirrors with low numbers,
+ by DNS and generally not documented anywhere except
+ for the master sites. However, official mirrors with low numbers
like 1-4, are usually <emphasis>Tier-1</emphasis>
- (this is just a rough hint, and there's no rule).
+ (this is just a rough hint, and there is no rule).
</para>
</sect2>
<sect2 id="mirror-where-where">
<title>Ok, but where should I get the stuff now?</title>
<para>
+ Under no circumstances should you mirror from <hostid>ftp.FreeBSD.org
+ </hostid>.
The short answer is: from the
- site, that is closest to you in Internet terms, or gives you
+ site that is closest to you in Internet terms, or gives you
the fastest access.
</para>
<sect3 id="mirror-where-simple">
@@ -798,7 +828,7 @@
</step>
<step>
<para>
- Check roughly those, which provide fastest access
+ Check for those which provide fastest access
(number of hops, round-trip-times)
and offer the services you intend to
use (like <application>rsync</application>
@@ -807,8 +837,8 @@
</step>
<step>
<para>
- Contact the admins of your chosen site, stating your
- request, and asking about their terms and
+ Contact the admins of your chosen site stating your
+ request and asking about their terms and
policies.
</para>
</step>
@@ -824,28 +854,36 @@
<para>
In general the description in <xref linkend="mirror-where-simple">
still applies. Of course you may want to put some
- weight on the fact, that your upstream should be of
+ weight on the fact that your upstream should be of
a low tier.
There are some other considerations about <emphasis>official</emphasis>
- mirrors, that are described in <xref linkend="mirror-official">.
+ mirrors that are described in <xref linkend="mirror-official">.
</para>
</sect3>
<sect3 id="mirror-where-master">
<title>I want to access the master sites!</title>
<para>
- If you have good reasons, and good prerequisites,
+ If you have good reasons and good prerequisites
you may want and get access to one of the
master sites. Access to these sites is
generally restricted, and there are special policies
for access. If you are already an <emphasis>official</emphasis>
- mirror, this certainly helps you getting access.
+ mirror this certainly helps you getting access.
In any other case make sure your country really needs another mirror.
- If it already has three or more, ask the &a.hubs; first.
+ If it already has three or more, ask the Zone Administrator
+ (<email>hostmaster at CC.freebsd.org</email>) or &a.hubs; first.
+ </para>
+ <para>
+ Whoever helped you become an <emphasis>official</emphasis> should
+ have helped you gain access to an appropriate upstream host, either
+ one of the master sites or a suitable Tier-1 site. If not you can
+ send email to <email>mirror-admin at FreeBSD.org</email> to request
+ help with that.
</para>
<para>
- There are just two master sites, one for the FTP fileset and
+ There are three master sites for the FTP fileset and
one for the CVS repository (the webpages and docs are
- obtained from CVS, so there is no need for it).
+ obtained from CVS, so there is no need for a master).
</para>
<sect4 id="mirror-where-master-ftp">
<title>ftp-master.FreeBSD.org</title>
@@ -865,10 +903,6 @@
access for the FTP contents, since they are
<emphasis>Tier-1</emphasis>-mirrors.
</para>
- <para>
- To get access to <hostid>ftp-master.FreeBSD.org</hostid>,
- you need to contact &a.ftp-master;.
- </para>
</sect4>
<sect4 id="mirror-where-master-cvsup">
<title>cvsup-master.FreeBSD.org</title>
@@ -956,13 +990,13 @@
<important>
<para>It is <emphasis>very</emphasis> important for a hub administrator, especially
Tier-1 hub admins, to check the
- <ulink url="http://www.FreeBSD.org/releases/4.6R/schedule.html">release schedule</ulink>
+ <ulink url="http://www.FreeBSD.org/releng/">release schedule</ulink>
for the next FreeBSD release. This is important because it will tell you when the
next release is scheduled
to come out, and thus giving you time to prepare for the big spike of traffic which follows it.
</para>
<para>
- It is also eminent that hub administrators try to keep their mirrors as up-to-date as
+ It is also important that hub administrators try to keep their mirrors as up-to-date as
possible (again, even more crucial for Tier-1 mirrors). If Mirror1 doesn't update for a
while, lower tier mirrors will begin to mirror old data from Mirror1 and thus begins
a downward spiral... Keep your mirrors up to date!
@@ -974,14 +1008,16 @@
<para>
An interesting question, especially, since the state
of being official comes with some benefits, like a much
- higher bill from your ISP, as more people will be using
- your site. Also it may be a key requirement, to get access
+ higher bill from your ISP as more people will be using
+ your site. Also it may be a key requirement to get access
to a master site.
</para>
<para>
- Before applying, please consider (again) if
+ Before applying please consider (again) if
another official mirror is really needed for
- your region. Ask on the &a.hubs;, if in doubt.
+ your region. Check first with your Zone Administrator
+ (<email>hostmaster at CC.FreeBSD.org</email>) or, if that
+ fails, ask on the &a.hubs;.
</para>
<para>Ok, here is how to do it:</para>
<procedure>
@@ -998,32 +1034,32 @@
</step>
<step>
<para>
- If everything works so far, contact the DNS admin, responsible
- for your region/country, and ask for a DNS entry for your
+ If everything works so far, contact the DNS admin responsible
+ for your region/country and ask for a DNS entry for your
site. The admin should able to be contacted via
- <email>hostmaster at cc.FreeBSD.org</email>, which
- <emphasis>cc</emphasis> being your country code/TLD
- again. Your DNS entry will look like described
+ <email>hostmaster at CC.FreeBSD.org</email>, where
+ <emphasis>CC</emphasis> is your country code/TLD.
+ Your DNS entry will be as described
in <xref linkend="mirror-where-organization">.
</para>
<para>
- If there is no subdomain delegated, yet, for your
- country, you probably need to contact
- <email>hostmaster at FreeBSD.org</email>,
- however, you can try the &a.hubs; first.
+ If there is no subdomain set up for your
+ country yet, you should contact
+ <email>mirror-admin at FreeBSD.org</email>
+ or you can try the &a.hubs; first.
</para>
</step>
<step>
<para>
- Then you can ask the &a.doc; or the &a.hubs;
- to add your mirror site to the mirror list in the
+ Whoever helps you get an official name should send email
+ to <email>mirror-admin at FreeBSD.org</email> so your site will be
+ added to the mirror list in the
<ulink url="http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook">FreeBSD
- Handbook</ulink>. Make sure you tell them the email address,
- to list as the maintainer of the site.
+ Handbook</ulink>.
</para>
</step>
</procedure>
- <para>This is it.</para>
+ <para>That is it.</para>
</sect2>
</sect1>
<sect1 id="mirror-statpages">
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-doc
mailing list