Child pages
  • GPU Computing (CS 205)
Skip to end of metadata
Go to start of metadata

Odyssey and GPU Computing on Odyssey

Before proceeding further, make yourself familiar with the basics of Odyssey and GPU computing on Odyssey:

https://rc.fas.harvard.edu/resources/odyssey-quickstart-guide/

https://rc.fas.harvard.edu/resources/documentation/gpgpu-computing-on-odyssey/

CUDA

Compiling and running CUDA code

1) Login to a node with a GPU, eg. holyseasgpu (for CS 205)

srun --pty --x11=first -p holyseasgpu --mem 4000 -t 0-5:00 -n 1 -N 1 --gres=gpu:1 bash

2) In a command window, type:

module load cuda/7.5.18-fasrc01

If your cuda program is 'test.cu', then you can compile with:

nvcc test.cu -o test

You can also use makefiles. Here is an example cuda code with a makefile:

main.cu

makefile

with these two files in the same directory, you can type, on the command line:

make

This will produce an executable 'square'

3) Run the cuda executable with the runscript:

--------------------------------

#!/bin/bash

#SBATCH -p holyseasgpu #Partition to submit to
#SBATCH -n 1 #Number of cores
#SBATCH --gres=gpu
#SBATCH -t 5 #Runtime in minutes
#SBATCH --mem-per-cpu=100 #Memory per cpu in MB (see also --mem)
#SBATCH --constraint=cuda-7.5

module load cuda/7.5.18-fasrc01

time ./square > output.txt 2> errors.txt

-------------------------------------------------

OpenACC

On Odyssey the PGI OpenACC compiler suite is installed in /n/seasfs03/IACS/cs205/pgi. To make the compilers (pgcc, pgc++, pgf90, etc.)

available in your path, add the following  lines to your .bashrc file (assumes you are using bash, which is the default shell):

export PGI=/n/seasfs03/IACS/cs205/pgi
export PATH=$PGI/linux86-64/17.10/bin:$PATH
export MANPATH=$MANPATH:$PGI/linux86-64/17.10/man
export LM_LICENSE_FILE=$LM_LICENSE_FILE:$PGI/license.dat

Once you add these to your ~/.bashrc, to make these take effect, you can do, in a terminal:

source ~/.bashrc

or

. ~/.bashrc
or you can log out and log back in.

Using OpenACC

You need to first compile your code (say code_acc.c or code_acc.f90) containing OpenACC (see below for example programs).

Note that pgcc and pgf90 should be available in your path for this to succeed (see above for instructions).

For c program:

pgcc -acc code_acc.c -Minfo=accel

For fortran program:

pgf90 -acc code_acc.f90 -Minfo=accel

will create an executable with name a.out. The option -Minfo=accel will display useful information on parallelization.


Slurm Script for running the job on odyssey:

--------------------------------

#!/bin/bash 

#SBATCH -N 1  #Number of nodes 

#SBATCH -p holyseasgpu  #Partition to submit to 

#SBATCH --ntasks-per-node 2

#SBATCH --gres=gpu:1

#SBATCH -t 5  #Runtime in minutes 

 ./a.out

---------------------------------

Links on OpenACC (with tutorials and sample OpenACC programs)

1) OpenACC example programs

    On Odyssey, you can find the OpenACC example programs in:

       $PGI/linux86-64/2016/examples/OpenACC

2) https://www.psc.edu/index.php/136-users/training/2430-xsede-hpc-workshop-december-6-2016-openacc

    Introduction to OpenACC (with sample code including Laplace) is in:

    https://www.psc.edu/images/xsedetraining/OpenACCDec2016/OpenACC_Introduction_To_OpenACC.PDF

3https://devblogs.nvidia.com/parallelforall/openacc-example-part-1/

4http://www.openacc.org/Sample_Codes

5) Introductory OpenACC tutorial (free, but requires an account):

   https://nvidia.qwiklab.com/quests/3?locale=en

PyCuda

Using PyCuda on Odyssey

The modules that need to be loaded for using pycuda on odyssey are as under (after sourcing new-modules.sh if you have not done that):

module load Anaconda/1.9.2-fasrc01

module load cuda/7.5-fasrc01

module load pycuda/2015.1.3-fasrc01

Pycuda examples come with the pycuda distribution. You can clone the pycuda repository as:

git clone --recursive http://git.tiker.net/trees/pycuda.git

The examples are in the directory: 

pycuda/examples/

Using the example "demo_elementwise.py" (demo_elementwise.py), a sample slurm script would be (pycuda_submit_wiki.sh)

-------------------------------

#!/bin/bash

#SBATCH -p holyseasgpu 

#SBATCH -n 4

#SBATCH --gres=gpu:1

#SBATCH --mem-per-cpu=4000

# Load required modules

source new-modules.sh

module load Anaconda/1.9.2-fasrc01

module load cuda/7.5-fasrc01

module load pycuda/2015.1.3-fasrc01

module list

# Run program

python demo_elementwise.py 

-------------------------------

(The example "dump_properties.py" (in pycuda/examples) will list the properties of cuda node on which slurm ran)

1) The following is one of the most important resources:

https://mathema.tician.de/software/pycuda/

2) Pycuda getting started tutorial:

https://documen.tician.de/pycuda/tutorial.html

3) More Pycuda examples (note many of these are for interactive work ,i.e. not suitable for slurm batch mode and may require

    additional tools not available on Odyssey).

https://wiki.tiker.net/PyCuda/Examples

PyOpenCL

Using PyOpenCL on Odyssey

The module that needs to be loaded for using PyOpenCL on odyssey is "pyopencl". Thus, one needs to do:

source new-modules.sh

 module load pyopencl

before PyOpenCL can be used. 

PyOpenCL examples come with the PyOpenCL distribution. You can clone the PyOpenCL repository as:

git clone --recursive http://git.tiker.net/trees/pyopencl.git

The examples are in the directory: 

pyopencl/examples/

Using the example "demo_elementwise.py", a sample slurm script would be:

---------------------------------

#!/bin/bash 

#SBATCH -J elementwise_pyopencl 

#SBATCH -o elementwise.out 

#SBATCH -e elementwise.err 

#SBATCH -p holyseasgpu 

#SBATCH -n 2 

#SBATCH --gres=gpu:1 

#SBATCH --mem-per-cpu=500

# Load required modules 

export PYOPENCL_CTX=''

source new-modules.sh

module load pyopencl

#Run program

python demo_elementwise.py 

--------------

1) https://mathema.tician.de/software/pyopencl/

2) https://wiki.tiker.net/PyOpenCL

3) https://www.khronos.org/opencl/

  • No labels