Installing Amber 11 on CentOS 4.3

Amber 11 on CentOS 4.3

This article will explain how to get Amber 11 ( http://ambermd.org ) to compile and run on my cluster with MPICH2. The systems I work with are old, well in terms of computers anyway. It consists of a cluster of PowerEdge 1950's running CentOS 4.3. Specifically:

• Dual Intel Xeon E5320 1.86GHz
• 4GB or 8GB 533MHz DDR2
• 146GB 15kRPM Hardrive
• CentOS 4.3
• gcc-4.1.2

Before we get started, some acknowledments are in order, many of the procedures posted herein are pulled from Mark Williamson [1], the MPICH2 readme [2] and from the Amber 11 and AmberTools 1.4 manuals [3].

[1] http://www.wmd-lab.org/mjw/howtos/compiling_gcc_4_4_4_under_RHEL4.txt

[2] http://www.mcs.anl.gov/research/projects/mpich2/index.php

[3] http://ambermd.org

The main reason for this, is that Amber 11 does not compile properly with gcc-4.1.2 which is available on CentOS 4.3. Therefore we must compile gcc-4.4.4, then MPICH2-1.2.1p1 and Amber 11 using it.

Setup

All the programs we setup here will be placed in /opt, and things are built in $BUILD as much as possible

$ BUILD=/scratch

Compiling gcc-4.4.4

gcc-4.4.4 requires gmp ( GNU MP, a library for abritrary precision arithmitic ). It is available on CentOS and must be installed prior to compilation.

 yum install gmp gmp-devel

This installs gmp-4.1.4.

Get gcc-4.4.4:

$ cd $BUILD

$ wget ftp://mirrors.kernel.org/gnu/gcc/gcc-4.4.4/gcc-4.4.4.tar.gz

$ sha1sum gcc-4.4.4.tar.bz2

45d1e7242a08cbb7a343ed2bf640a5afd0ebe098 gcc-4.4.4.tar.bz2

Untar it.

$ tar -jxf gcc-4.4.4.tar.bz2

Gcc requires MPFR 2.3.2+ for it compile. MPFR is not available on CentOS 4.3, so we must provide it within the gcc source directory. The highest version of MPFR that is recommended with our version of gmp is MPFR-2.3.2

Get MPFR 2.3.2

$ wget http://www.mpfr.org/mpfr-2.3.2/mpfr-2.3.2.tar.bz2

$ sha1sum mpfr-2.3.2.tar.bz2

37dbd478e1c136f37cf4c68eb1522e86e2307288 mpfr-2.3.2.tar.bz2

The mpfr source must be placed within the gcc source directory in a directory

named 'mpfr'.

$ tar -jxf mpfr-2.3.2.tar.bz2

$ sha1sum mpfr-2.3.2.tar.bz2

37dbd478e1c136f37cf4c68eb1522e86e2307288 mpfr-2.3.2.tar.bz2

$ mv mpfr-2.3.2 gcc-4.4.4/mpfr

Create a build directory for building gcc outside the main source directory.

$ mkdir build

$ cd build

Run the gcc configure script from the build directory. We only need c,c++ and

fortran, so we will enable only them.

$ ../gcc-4.4.4/configure --enable-languages=c,c++,fortran --prefix=/opt/gcc-4.4.4

This is running on an 8 core node, so make with '-j8' to enable compililation on

all cores.

$ make -j8

This takes about 16 minutes.

Install the compiler.

$ sudo make install

Compilation MPICH2

Setup the environment to use the newly installed gcc-4.4.4.

$ export PATH=/opt/gcc-4.4.4/bin:$PATH

$ export LD_LIBRARY_PATH=/opt/gcc-4.4.4/lib:/opt/gcc-4.4.4/lib64:$LD_LIBRARY_PATH

$ export MANPATH=/opt/gcc-4.4.4/man:$MANPATH

Set the fortran environment variables so that MPICH2 enables fortran modules.

$ export F90=gfortran

$ export F77=gfortran

Go back to the build directory and get mpich2 source code.

$ cd $BUILD

$ wget http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/1.2.1p1/mpich2-1.2.1p1.tar.gz

$ sha1sum mpich2-1.2.1p1.tar.gz

26bd5620b45bc654aa3945755d11450be63d650c mpich2-1.2.1p1.tar.gz

$ tar -zxf mpich2-1.2.1p1.tar.gz

Configure and make.

$ cd mpich2-1.2.1p1

$ ./configure --prefix=/opt/mpich2-1.2.1p1 2>&1 myconf.log

$ make

This takes about 4 minutes.

And finally Install

$ sudo make install

Setup the environment to use our newly compilied MPICH2 install.

$ export PATH=/opt/mpich2-1.2.1p1/bin:$PATH

$ export LD_LIBRARY_PATH=/opt/mpich2-1.2.1p1/lib:$LD_LIBRARY_PATH

$ export MANPATH=/opt/mpich2-1.2.1p1/share/man:$MANPATH

Do some quick MPICH2 envrioment tests. See the mpich2-1.2.1p1 README[4] for more details.

$ which mpd

/opt/mpich2-1.2.1p1/bin/mpd

$ which mpiexec

/opt/mpich2-1.2.1p1/bin/mpiexec

$ mpd &

$ mpdtrace

compute-x-y

$ mpdallexit

[4] https://svn.mcs.anl.gov/repos/mpi/mpich2/tags/release/mpich2-1.2.1/README.vin

Amber 11

Unpack Amber

$ cd $BUILD

$ sha1sum Amber11.tar.bz2 AmberTools-1.4.tar.bz

fdf3e840da58a8ae4f13cdf7d9ef037c4c69fa67 Amber11.tar.bz2

ef0f9a44c8767ed35f942fd41afc636b67da0389 AmberTools-1.4.tar.bz2

$ tar -jxf Amber11.tar.bz2

$ tar -jxf AmberTools-1.4.tar.bz2

Get the latest bugfixes and patch the source.

$ wget http://ambermd.org/bugfixes/11.0/bugfix.all

$ mv bugfix.all amber11.p1-6.patch

$ cd amber11

$ patch -N -p0 < ../amber11.p1-6.patch patching file src/sander/qm_mm.f patching file src/sander/nonbond_list.f patching file src/pmemd/src/mdin_ctrl_dat.fpp patching file src/pmemd/src/cuda/gpu.cpp patching file src/pmemd/src/cuda/gpu.cpp patching file src/sander/makedepend patching file src/sander/Makefile patching file src/sander/mdread.f

Amber has a poor compile/make setup. It has a bad habit of hardcoding paths

into libraries and executables. This forces the compilation of Amber in the

destination directory. In production environments such as mine, this is not

ideal, but it is something we must deal with. In order to do a cheap workaround

I copy the amber11 tools to my target directory and do some permission magic so

I can compile as a user. Then change permissions back so I can rollout to my

cluster.

Copy the amber11 directory to the target directory. I name it amber11p6 to

indicate that it has upto bugfix 6 applied. And change the permissions to my

user account.

$ sudo cp -r $BUILD/amber11 /opt/amber11p6

$ sudo chown -R 1000:1000 /opt/amber11p6

Now start the Amber 11 configure/make process

$ cd /opt/amber11p6

$ export AMBERHOME=`pwd`

$ cd AmberTools/src

$ ./configure -noX11 gnu

Make AmberTools. Unfortuanely the Makefiles don't play nice with parallel make

options available from the make tool. So we must do it serially. This takes

about 9 minutes.

$ make install

Here comes the fun part. We'll start the testing of AmberTools while compiling

amber11.

Start AmberTools testing. This takes about 1/2 hour.

$ cd ../test

$ make test

While this is testing, open a new window and setup the enviroment. I can do this

because I'm using an 8 core node. Each test is single threaded.

$ cd /opt/amber11p6

$ export PATH=/opt/mpich2-1.2.1p1/bin:/opt/gcc-4.4.4/bin/:$PATH

$ export LD_LIBRARY_PATH=/opt/mpich2-1.2.1p1/lib:/opt/gcc-4.4.4/lib:/opt/gcc-4.4.4/lib64:$LD_LIBRARY_PATH

$ export AMBERHOME=`pwd`

In window 2, compile amber11. This takes a 3 minutes.

$ cd src

$ make install

Now, in Window 2 run the amber11 tests. This takes about 75 minutes.

$ cd ../test

$ make test

Close the second window, we don't need it anymore.

When both serial tests are done, we can move on to

compiling the parallel versions of Amber11. This takes about

5 minutes.

$ cd $AMBERHOME/AmberTools/src

$ ./configure -noX11 -mpi gnu

$ cd ../../src

$ make clean

$ make parallel

Setup a simple Parallel environment to test the MPI environment. We'll try doing

it with 2 and 4 processors. Testing beyond this causes PIMD tests to hang. This

is a known bug and is being worked on as of this writing. ( July 2010 )

2 Cores, about 24 minutes.

$ mpd &

$ export DO_PARALLEL='mpiexec -n 2'

$ cd ../test

$ make test.parellel

4 Cores, about 16 minutes.

$ export DO_PARALLEL='mpiexec -n 4'

$ make test.parellel

All done for now

$ mpdallexit

Taring everything up

This is pretty easy, just use tar.

$ cd /

$ tar -cP --owner=root --group=root -f ~/gcc-4.4.4-x86_64.tar /opt/gcc-4.4.4

$ tar -cP --owner=root --group=root -f ~/mpich2-1.2.1p1-x86_64.tar /opt/mpich2-1.2.1p1

Amber and AmberTools do not need all the compiliation directories to work.

So we only tar the files we need.

$ tar -cP --owner=root --group=root -f ~/amber11p6-x86_64.tar \

 /opt/amber11p6/AmberTools/{bin,include,exe,lib} \

/opt/amber11p6/{bin,include,exe,lib,dat}

Now compress them

$ cd ~

$ gzip -9 {gcc-4.4.4,mpich2-1.2.1p1,amber11p6}-x86_64.tar

And we are done.