XStream is a Linux GPU cluster running Red Hat Enterprise Linux 6.9.
Modules provide a convenient way to dynamically change the users’ environment through modulefiles. This includes easily adding or removing directories to the PATH environment variable.
Lmod is used as a replacement to the original module command. For more information, please take a look at the Lmod user guide
Check the modules that can be directly loaded using:
You can list all available modules using the
Using the full name will give you details on how to load the module by listing any required dependencies:
Also, you can use
module list to see currently loaded packages.
To load packages:
Note: order matters when loading modules (eg. a same package can be available from different compilers).
Again, take a look at the Lmod user guide for more info.
Compiler toolchains are basically a set of compilers together with libraries that provide additional support that is commonly required to build software. In the HPC world, this usually consists of an library for MPI (inter-process communication over a network), BLAS/LAPACK (linear algebra routines) and FFT (Fast Fourier Transforms).
List of available compiler toolchains on XStream:
To load the FOSS toolchain:
To load the Intel compiler toolchain:
Or you can just use the following command to load the default Intel compiler toolchain:
Note: loading a toolchain will offer you additional software to load through module. Check
module avail once your preferred
compiler toolchain is loaded.
The 2016 edition of the Intel compilers is available on XStream as a separate module (it is NOT a real compiler toolchain for now).
You can load it using:
The following versions of CUDA are available on XStream: 6.5.14, 7.0.28, 7.5.18 (default), 8.0.44 and 8.0.61.
For example, to load CUDA 8.0.61, please use:
The Nvidia driver version is currently v375.66 on all nodes.
This section provides an overview of how to use pre-installed software managed by the Research Computing team in collaboration with XStream users.
The foss toolchain includes gcc-based OpenMPI.
ATLAS (Automatically Tuned Linear Algebra Software) is the application of the AEOS (Automated Empirical Optimization of Software) paradigm, with the present emphasis on the Basic Linear Algebra Subprograms (BLAS), a widely used, performance-critical, linear algebra kernel library.
The NVIDIA CUDA Deep Neural Network library (cuDDN) is a GPU-accelerated library of primitives for deep neural networks.
GROMACS is a versatile package to perform molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles.
NAMD is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems.
OpenMM is a high performance toolkit for molecular simulation. On XStream, it is compiled against the foss toolchain (GCC 4.9.2) and CUDA 7.0.28.
PostgreSQL is an open source relational database management system (DBMS) developed by a worldwide team of volunteers.
PG-Strom is an extension of PostgreSQL designed to off-load several CPU intensive workloads to GPU devices, to utilize its massive parallel execution capability.
$WORK/postgres/data/postgresql.conf and add the following line to load the PG-Strom extension:
Start the PostgreSQL server:
Connect using your login name and the password postgres and create the pg_strom extension:
Stop the PostgreSQL server:
Please always use PostgreSQL within a job, not directly on the login nodes.
PyTorch is a new deep learning framework with GPU acceleration. The early release 0.2.0 is now available on XStream as a special module that will load all required dependencies (Python 2.7.9, CUDA 8.0.41, cuDNN 5.1 and MAGMA 2.2.0).
Usage (Python 2.7):
R is a free software environment for statistical computing and graphics.
RStudio IDE is a powerful and productive user interface for R.
TensorFlow is an Open Source Software Library for Machine Intelligence originally developped by Google with CUDA 8 support.
Usage (Python 2.7):
Usage (Python 3.6):
Note: tensorflow is a special module that will load the foss toolchain automatically.
Newer versions of TensorFlow should be run in Singularity containers.
TeraChem is general purpose quantum chemistry software designed to run on NVIDIA GPU architectures under a 64-bit Linux operating system.
Usage example without MPI support:
Usage example with MPI support:
Torch is a scientific computing framework with wide support for machine learning algorithms that puts GPUs first. It has been compiled against the foss toolchain (GCC based) and cuDNN 4.0 (CUDA 7.5.18).
Note: torch is a special module that will load CUDA, cuDNN and the foss toolchain automatically.
Currently available third-party packages are:
VMD is a molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3-D graphics and built-in scripting.
VMD on XStream is built against CUDA 7.0.28 and Nvidia OptiX 3.8.0, enabling the TachyonL-OptiX GPU-accelerated ray tracing renderer available in VMD 1.9.2. At least the following features should also be available: ACTC library support, collective variables, Python support, Pthreads, NetCDF, ImageMagick, ffmpeg and NetPBM.
Prerequisite: use ssh X11 forwarding by adding
-X to your ssh command when connecting to XStream.
For non-computationally expensive tasks with VMD, you may launch VMD from a login node:
To perform computationally expensive tasks with VMD, please launch VMD using srun with the X11 option as shown here (example with 1 task, 4 CPUs and 4 GPUs):
Have questions? Feel free to contact Research Computing Support.