Nextflow

This documentation guides you through running your existing Nextflow pipeline on MOGON and making use of its computing resources.

Your pipeline probably already runs locally, so getting it onto MOGON is mainly about making it work with a shared computing environment. Jobs may have to wait in a queue and the initial setup can take a little patience. But it will be worth it, as in return, MOGON gives your pipeline access to many CPU cores and more computing resources, allowing you to process more datasets and therefore to be more time efficient.

You know your pipeline and your data, this documentation helps you connect them to MOGON. If something does not work as expected, we will figure it out together along the way.

Usage

Nextflow configuration#

To run your pipeline on MOGON, Nextflow needs to know that jobs should be submitted through Slurm.

Create and open a nextflow.config file in your working directory, e.g. with:

nano nextflow.config

The following configuration tells Nextflow to submit processes through Slurm and ensures that each step of your workflow is sent to the right partition based on the ressource needs specified in the pipeline. It also enables the use of the container system apptainer, so dependencies can be pulled by Nextflow as specified in the pipeline.

apptainer.enabled = true

process {

    executor = 'slurm'

    queue = {
        if (task.cpus < 32 &&
            task.memory <= 256.GB) {
            'ki-smallcpu'
        }

        else if (task.memory <= 512.GB) {
            'ki-parallel'
        }

        else if (task.memory <= 1016.GB) {
            'ki-largemem'
        }

        else if (task.memory <= 1992.GB) {
            'ki-hugemem'
        }
    }
}

Running the pipeline on MOGON#

You can run a workflow directly from the command line on the login node or submit it via sbatch.

• When running the workflow directly from the command line, Nextflow runs in your current terminal session and the workflow progress is displayed directly in the terminal.
• When submitting the workflow via sbatch, Nextflow runs as a SLURM batch job on a core on MOGON. The workflow progress is written to the corresponding SLURM output file and can be monitored there, as descripted below.
• Using sbatch is recommended for longer-running pipelines. The Nextflow process then runs independently of your login session and continues even after you are logged out.

To see which Nextflow versions are available, run:

module avail nextflow

Choose the required version and load it using the commands described below.

Running the pipeline via CLI#

For shorter pipelines, development, or testing, you can run Nextflow directly from the command line. In this case, load the required modules:

module load tools/Nextflow/26.04.6  # choose the version you need (check available options; see above)
module load tools/Apptainer/1.3.4-GCCcore-13.3.0

and execute the nextflow run command directly:

nextflow run <URL to github repository> OR <path/to/your/workflow.nf> \
   -c /path/to/your/nextflow.config \
   --input /path/to/your/mini-pipeline/data \
   --outdir /path/to/your/desired/output-directory

Be aware that the login node automatically logs users out after some time of inactivity. A Nextflow process running directly in the terminal may therefore be terminated when the session ends.

Running the pipeline via sbatch#

To submit your pipeline to MOGON, create a SLURM job script. This script loads the necessary modules and starts your Nextflow pipeline.

Create and open a new script.sh file, e.g. with:

nano script.sh

Copy the following into the file and:

1. Adjust all values marked with <...>.
2. Check that the correct Nextflow version is loaded.
3. Adjust the paths to match the location of your files.

# 1. Mogon configuration
#SBATCH -J <job name>        # Job name
#SBATCH -M <cluster name>    # Cluster name
#SBATCH -A <account name>    # Account name
#SBATCH -p <partition name>  # Partition name
#SBATCH -t <time in minutes> # Time in minutes
#SBATCH -n 1                 # Number of tasks
#SBATCH -c 1                 # Number of CPUs
#SBATCH --mem=20G            # Memory per node

# 2. Load required modules

module load tools/Nextflow/26.04.6  # choose the version you need (check available options; see above)
module load tools/Apptainer/1.3.4-GCCcore-13.3.0

# 3. Run the pipeline

nextflow run <URL to github repository> OR <path/to/your/workflow.nf> \
   -c /path/to/your/nextflow.config \
   --input /path/to/your/mini-pipeline/data \
   --outdir /path/to/your/desired/output-directory

The SLURM settings determine the resources allocated to the job only executing the nextflow run command. The resources required by the individual processes of your pipeline are managed by the pipeline itself, so you do not need to configure them in the SBATCH script.

However, make sure that the time limit specified with #SBATCH -t is long enough for the entire pipeline to complete. The SLURM job running Nextflow acts as the workflow orchestrator and remains active for the entire duration of the pipeline.

To run your pipeline, just submit the Slurm job to MOGON:

sbatch script.sh

Your job is now submitted and will run when the requested resources become available. You can check its current status with:

squeue -u <your username>

Monitoring pipeline progress#

The progress of the Nextflow pipeline is written to the SLURM output file. The output file is identified using the job ID of the SLURM job submitted with sbatch.

You can monitor the pipeline in two ways:

1. Continuous monitoring using:

tail -f slurm-<jobid>.out

This continuously displays new output as the Nextflow pipeline progresses. Exit tail with Ctrl + C.

2. Check current progress using:

cat slurm-<jobid>.out

This displays the current contents of the output file. You can run this command at any time to check the current pipeline progress.