Job Examples
How to use Slurm for submitting batch jobs to MOGON
Single Core Job
This script runs on a single CPU on MOGON KI’s ki-smallcpu partition.
#!/bin/bash
#========[ + + + + Requirements + + + + ]========#
#SBATCH --account=<ki-project>
#SBATCH --job-name=test-serial
#SBATCH --partition=ki-smallcpu
#SBATCH --comment="a comment to keep track of things"
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --mem=300M
#SBATCH --time=00:30:00 # 30 minutes
#SBATCH --output=mysimplejob.%j.out
#========[ + + + + Environment + + + + ]========#
# Load all necessary modules in the script to ensure a consistent environment.
module purge
module load gcc/6.3.0
#========[ + + + + Job Steps + + + + ]========#
# Launch the executable
srun --hint=nomultithread <myexecutable>Full Node Job – Threaded Application
The following script will launch one task using 128 cores (please note that most applications do not scale that far). Since we reserve a whole node, we have access to the node’s complete memory.
#!/bin/bash
#========[ + + + + Requirements + + + + ]========#
#SBATCH --account=<nhr-project>
#SBATCH --job-name=test-full-node
#SBATCH --comment="now running on MOGON NHR"
#SBATCH --partition=parallel
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=128
#SBATCH --time=00:30:00
#SBATCH --output=test-full-node.%j.out
#========[ + + + + Environment + + + + ]========#
# Load all necessary modules in the script to ensure a consistent environment.
module load gcc/6.3.0
#========[ + + + + Job Steps + + + + ]========#
# Launch the executable with one task distributed on 128 cores:
srun <myexecutable>Full Node Job – MPI Application
This script requests 256 MPI tasks on two nodes. The job will have access to the nodes’ complete memory.
#!/bin/bash
#========[ + + + + Requirements + + + + ]========#
#SBATCH --account=<ki-project>
#SBATCH --partition=ki-parallel
#SBATCH --job-name=test-multi-node
#SBATCH --comment="now back on MOGON KI again"
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=128
#SBATCH --time=00:30:00
#SBATCH --output=test-multi-node.%j.out
#========[ + + + + Environment + + + + ]========#
module purge
module load <appropriate module(s)>
#========[ + + + + Job Steps + + + + ]========#
srun <myexecutable>Hybrid MPI-OpenMP Job
Whereas MPI applications frequently adhere to the standard MPI idea of parallelization by multiprocessing and exchanging messages between the created processes, hybrid applications use internally threaded processes (e.g. either by means of MPI tasks or OpenMP threads).
For this example we assume you want to run GROMACS
- on 2 nodes (128 CPUs per node)
- with 32 MPI tasks in total,
- using 8 CPUs for OpenMP threads per MPI task.
The job script could look like this:
#!/bin/bash
#========[ + + + + Requirements + + + + ]========#
#SBATCH --account=<nhr-project>
#SBATCH --partition=parallel
#SBATCH --job-name=my-gromacs-job
#SBATCH --comment="hybrid job on MOGON NHR"
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=16
#SBATCH --cpus-per-task=8
#SBATCH --time=00:30:00
#SBATCH --output=%x.%j.out
#SBATCH --error=%x.%j.err
#SBATCH --mail-type=END,FAIL
#SBATCH --mail-user=<username@uni-mainz.de>
#========[ + + + + Environment + + + + ]========#
module purge
module load bio/GROMACS
export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK}
export OMP_PROC_BIND=close
export OMP_PLACES=threads
#========[ + + + + Diagnostics + + + + ]========#
echo "=== Job ${SLURM_JOB_ID} (${SLURM_JOB_NAME}) on $(hostname) ==="
echo "Allocated nodes : $SLURM_JOB_NODELIST"
echo "OMP threads per rank : $OMP_NUM_THREADS"
echo "MPI ranks : $SLURM_NTASKS"
module list
#========[ + + + + Job Steps + + + + ]========#
srun gmx_mpi mdrun -ntomp ${SLURM_CPUS_PER_TASK} -deffnm emRelated Topics
Using GPUs
For examples of how to use the GPU nodes, take a look here.