Personal Modulefiles

This tutorial is for users wishing to create modulefiles for their own software. The reasons are simple:

  1. Install a newer or older version of open-source software than is currently available.
  2. Easily change the version of applications or libraries under their own development.
  3. Better documentation for what software is available.

In the following tutorial, we will install a small tool called zlib and create a modulefile for demonstration.

First, create two folders in your /home directory named software and modulefiles.

[jzhan61@clogin01 ~]$ pwd
/home/jzhan61
[jzhan61@clogin01 ~]$ mkdir software modulefiles

Change into the software folder, create a subdirectory named ‘zlib’, and cd into it. 

[jzhan61@clogin01 ~]$ cd software/
[jzhan61@clogin01 software]$ mkdir zlib
[jzhan61@clogin01 software]$ cd zlib/
[jzhan61@clogin01 zlib]$ pwd
/home/jzhan61/software/zlib

Download the source file to this directory, and unzip it.

#Screen output of the following commands are not shown here to save space
[jzhan61@clogin01 zlib]$ wget https://zlib.net/zlib-1.2.11.tar.gz
[jzhan61@clogin01 zlib]$ tar zxvf zlib-1.2.11.tar.gz

Create a folder named install as installation destination.

[jzhan61@clogin01 zlib]$ pwd
/home/jzhan61/software/zlib
[jzhan61@clogin01 zlib]$ mkdir install

Change into the source code directory and compile the software using configure, make and make install. 

[jzhan61@clogin01 zlib]$ pwd
/home/jzhan61/software/zlib
[jzhan61@clogin01 zlib]$ ls
install  zlib-1.2.11  zlib-1.2.11.tar.gz
[jzhan61@clogin01 zlib]$ cd zlib-1.2.11/

#Screen output of the following commands are not shown here to save space
[jzhan61@clogin01 zlib-1.2.11]$ ./configure --prefix=/home/jzhan61/software/zlib/install
[jzhan61@clogin01 zlib-1.2.11]$ make
[jzhan61@clogin01 zlib-1.2.11]$ make install

Now the software has been installed at the destination folder.

[jzhan61@clogin01 zlib-1.2.11]$ cd /home/jzhan61/software/zlib/install/
[jzhan61@clogin01 install]$ ls
include  lib  share

Next, let’s create the module file itself. First, create a zlib folder under the modulefiles directory.

[jzhan61@clogin01 ~]$ mkdir /home/jzhan61/modulefiles/zlib
[jzhan61@clogin01 ~]$ cd /home/jzhan61/modulefiles/zlib
[jzhan61@clogin01 zlib]$

Now create a file named 1.2.11.lua in this folder with the following contents

help([[zlib 1.2.11

This module loads the zlib library.

]])
whatis("Name: zlib")
whatis("Version: 1.2.11")
whatis("Category: library, runtime support")
whatis("Keywords: System, Library")
whatis("URL: http://www.zlib.net/")

prepend_path("LD_LIBRARY_PATH","/home/jzhan61/software/zlib/install/lib")
prepend_path("MANPATH","/home/jzhan61/software/zlib/install/share/man")
prepend_path("INCLUDE","/home/jzhan61/software/zlib/install/include")

The final step, type module use /home/jzhan61/modulefiles to enable this module file. 

[jzhan61@clogin01 zlib]$ module use /home/jzhan61/modulefiles

Let’s check what has changed in the module av command.

[jzhan61@clogin01 zlib]$ module av

---------------------------------------- /home/jzhan61/modulefiles ----------------------------------------
   zlib/1.2.11

------------------------------------ /apps/lmod/lmod/modulefiles/Core -------------------------------------
   GEM/1.1              cmake/3.18.2             lmod
   MATLAB/R2020a        intel/2020.2.254         openmpi/gcc/4.8.5/4.0.3
   R/4.0.2       (D)    intelmpi/20200624        openmpi/intel/2020.2.254/4.0.3
   R/4.0.3              julia/1.0.5              python/3.8
   SAS/9.4              julia/1.5.0       (D)    settarg

Note aside from the system’s core modules, now I have a personal module named zlib/1.2.11. We can load it the same way as other system modules.

[jzhan61@clogin01 zlib]$ module load zlib/1.2.11
[jzhan61@clogin01 zlib]$ ml

Currently Loaded Modules:
  1) zlib/1.2.11



Notes:
1. Add the line module use ~/modulefiles to your ~/.bashrc file so you don’t have to retype it every time you login.
2. Installation procedures for different software packages could be slightly different. Read the software installation manual for help.
3. You don’t have to be an expert in lua programming to create module files. Most of the time you can simply copy/paste and make modifications.
4. If you want to install a package for your research group, then install it in the /project folder instead of /home where everyone has read/write access.