[CFT] Hadoop preliminary port

Bleakwiser traebarlow at gmail.com
Mon Feb 13 15:26:32 UTC 2012

*FreeBSD 9.0 / Hadoop 1.0.0 Install Guide From Preliminary Port*

After installing FreeBSD 9.0 AMD64 I used 'portsnap fetch' to fetch the
latest ports tree, installed Portmaster and with it the Diablo-JDK16 port,
wget, svn, rsync, Tomcat7 and Hadoop 1.0.0 .

There are plenty of guides online how to get all of the former ports
installed, however Hadoop has not been well documented in a 'newb
sensative' manner so I decided to take it upon myself to get some more
people involved in testing Hadoop on FreeBSD.

Some preliminary knowledge you're going to need to know. The science of
UIDs and GIDs, the history and usefullnes of patch files (.diff), and how
to use Subversion. Let's jump right in, shall we?

The Ports Tree keeps a list of UIDs and GIDs for user and group creation
when a corrosponding app in the list is installed. There is no exception to
Hadoop, except that Hadoop's listings are not included in the standard
Ports Tree by default. So they need to be added manually, luckily our
friend Clement has done that for us and has created a patch file (.diff)
for your convenience using the popular 'patch' utility. Before manually
updating your ports tree Confucius say wise man always update first...
however it's not really necessary.

localhost# portsnap update
localhost# cd /usr/ports
localhost# wget http://people.freebsd.org/~clement/hadoop/hadoop-1.0.0.diff
localhost# patch < hadoop-1.0.0.diff

Congradulations, your Ports Tree is now has the necissary information to
create the necissary users and groups Hadoop will need during
installation (that is Ports does, Hadoop needs the info after the install).

Now we can add the Hadoop directory to the ports tree. I trust that you
have already installed Subversion. If not, go ahead and do that now. It's
in devel/subversion. Not being the focus of this guide you'll have to
google up one of the other dozens of guides on how to install it (or use
common sense, it's nothing special).

You need to be in the /usr/ports/devel directory so that Subversion will
create the Hadoop directory in the proper part of the Ports Tree saving you
from having to 'cp' or 'mv' it later.
localhost# cd /usr/ports/devel
localhost# svn checkout https://svn.redports.org/clement/devel/hadoop/

Now you should have a ./hadoop directory in ./devel. Lets check it.
localhost# cd hadoop/
localhost# ls

Amongst a handfull of other files you should see a Makefile and a files/
directory. Now to the fun part. I like to install all my ports from
/usr/ports so that way Portmaster can take advantage of the shell's
autocomplete functions.

localhost# cd /usr/ports
localhost# portmaster devel/hadoop

Don't quote me on this, but unless you have a good reason you shouldn't
need to stray from any of the default configuration options.

If all goes well you should see something like....

Total time: 1 second
===>  Installing for hadoop-1.0.0
===>   Generating temporary packing list
===>  Checking if devel/hadoop already installed
===> Creating users and/or groups.
Creating group `hadoop' with gid `954'.
Creating user `hadoop' with uid `954'.
===> Installing rc.d startup script(s)
=> Creating RUNDIR /var/run/hadoop... => Creating LOGDIR /var/log/hadoop...
===> Correct pkg-plist sequence to create group(s) and user(s)
===>   Registering installation for hadoop-1.0.0
===>  Cleaning for hadoop-1.0.0
===>>> Installation of devel/hadoop (hadoop-1.0.0) complete

Now all that is left is to configure the Hadoop server as either a
Standalone, Pseudo-Distributed or Distributed Operation Server. So my work
is done, the official apache.org site takes it from here. Visit the link
below if you wish to continue on down the rabbit hole.


View this message in context: http://freebsd.1045724.n5.nabble.com/CFT-Hadoop-preliminary-port-tp4677071p5479564.html
Sent from the freebsd-ports mailing list archive at Nabble.com.

More information about the freebsd-ports mailing list