Java

Java is a widely used, object-oriented programming language that offers platform independence through the Java Virtual Machine (JVM). Its runtime environment provides features such as Just-in-Time (JIT) compilation, automatic memory management, and support for concurrent and parallel programming. These characteristics make Java a potential choice for High-Performance Computing (HPC) applications.

Different approaches to using Java in HPC environments have been explored, particularly regarding parallelization, performance optimization, and integration with HPC systems. For further information, look into the official Java documentation or see this publication on Java for High-Performance Computing.

Usage#

To access Java on MOGON, check which Java modules are available and load the desired version. If no version is specified, the default version is loaded.

module avail lang/Java
module load lang/Java

To check the installed Java version and the location of the executables, use:

java --version
javac --version
which java
which javac

Where, java is the Java launcher used to start Java applications on the JVM, while javac is the Java compiler used to compile Java source code.

Compiling and running Java programs#

To use Java on MOGON, create a .java source file, compile it using the Java compiler (javac), and execute the resulting program using the Java runtime (java).

For example, create a file named HelloMogon.java:

public class HelloMogon {
    public static void main(String[] args) {
        System.out.println("Hello MOGON!");
    }
}

Compile the program:

javac HelloMogon.java

This generates a compiled bytecode file named HelloMogon.class.

Execute the program:

java HelloMogon

Parallelization and CPU allocation#

Java supports multithreading through its built-in thread API. Threads can be created and managed directly or through higher-level concurrency utilities such as the ExecutorService and thread pools.

Java threads and SLURM#

Java thread management and Slurm CPU allocation are handled separately. The JVM can detect the number of processors available to the process based on the resources allocated by Slurm. However, this does not automatically determine the number of application threads. Application threads are created and managed by the Java application itself.

You can make the CPU allocation visible by checking how many processors the JVM detects as available within your Slurm allocation. Add the following code to your Java program:

System.out.println("Available processors: " + Runtime.getRuntime().availableProcessors());

Java Overhead#

As Java creates additional threads for JVM maintainance (e.g. garbage collection), it is advisable to reserve an additional core for the Java overhead. I.e., if you have a single-threaded Java application, you should reserve a second CPU slot by using the -c 2 option in your submission script to sbatch.

The following tutorial provides a practical introduction to running Java applications in parallel on MOGON. For more detailed information, refer to the official Java documentation.

Fundamentals of ExecutorService

Fundamentals of Java threads

Running Java with Slurm#

To build onto our “Hello Mogon!” example, we now create a file HelloMogonParallel.java for a simple parallel example using the ExecutorService as a thread manager. We declare a fixed ThreadPool with 4 java threads that are then getting used to say “Hello!”.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class HelloMogonParallel {
    public static void main(String[] args) {

        ExecutorService executor = Executors.newFixedThreadPool(4);

        for (int i = 1; i <= 4; i++) {
            final int threadNumber = i;

            executor.submit(() -> {
                System.out.println("Hello from Thread " + threadNumber);
            });
        }

        executor.shutdown();
    }
} 

Then create a job script:

#!/bin/bash
#SBATCH --job-name=java-threads
#SBATCH --partition=ki-smallcpu
#SBATCH --account=<ki-project>
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=5
#SBATCH --mem=5G
#SBATCH --time=00:05:00
#SBATCH --output=java-threads_%j.out
#SBATCH --error=java-threads_%j.err

#========[ + + + + Environment + + + + ]========#
# Load all necessary modules to ensure a consistent environment.
module purge
module load lang/Java

#========[ + + + + Job Steps + + + + ]========#
# compile
javac HelloMogonParallel.java

# run
srun java HelloMogonParallel

Submit the job script and look at the output, it should contain the following:

Hello from Thread 1
Hello from Thread 3
Hello from Thread 4
Hello from Thread 2

Using external Libraries#

To use external libraries, you can place their JAR files in your project directory and include them in the classpath when compiling and running your Java program. Compile and execute with:

javac -cp ".:path/to/your/package.jar" Main.java
java -cp ".:path/to/your/package.jar" Main

The following example demonstrates this process using a custom Java class packaged into a JAR file and then imported into another program. The same approach applies to external libraries: you can download the required JAR files to your local computer and transfer them to your MOGON working directory.

Creating JAR Libraries#

We define a custom class Message.java:

public class Message {
    public static String getMessage(int i) {
        return "Hello from an external JAR library used by thread " + i + "!";
    }
}

To transform it into a JAR file, we need to compile it and create the file:

javac Message.java
jar cf message.jar Message.class

Using JAR Libraries#

To use an external JAR library, we compile and run the Java program with a reference to the JAR file in the classpath.

We adapt our previous example to use the external Message class to print a greeting. Create a file named HelloMogonJAR.java:

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class HelloMogonJAR {
    public static void main(String[] args) {

        ExecutorService executor = Executors.newFixedThreadPool(4);

        for (int i = 1; i <= 4; i++) {
            final int threadNumber = i;

            executor.submit(() -> {
                System.out.println(Message.getMessage(threadNumber));
            });
        }

        executor.shutdown();
    }
}

In your Slurm job script, both the compilation and execution commands must include the JAR file in the classpath:

#========[ + + + + Job Steps + + + + ]========#

# Compile
javac -cp ".:message.jar" HelloMogonJAR.java

# Run
java -cp ".:message.jar" HelloMogonJAR

Memory#

To run Java on MOGON, you need to reserve much more memory than you think. The Java Virtual Machine (JVM) manages memory independently of the memory allocation specified in Slurm.

One example: Just calling java -version with a specified maximum heap size (1GB in this example), you need to reserve sufficient memory or Java will not be able to run correctly. Note that for larger heap sizes, you just need to keep the offset large enough.

#!/bin/bash

# for multithreaded JAVA programs it might(!) make sense to occupy a full node
#SBATCH -p short   # MOGON I
#SBATCH -p smp     # MOGON II
#SBATCH -A <mogon-project>
#SBATCH --mem=1800M

module load lang/Java

export MALLOC_ARENA_MAX=4
srun  java -Xms1G -Xmx1G -version

The -Xms and -Xmx options control the initial and maximum Java heap sizes. However, the Java heap is only one part of the JVM’s total memory consumption. Additional memory is required for JVM internals, thread stacks, JIT compilation, and native allocations.

Therefore, the Slurm memory allocation (–mem) must be larger than the maximum heap size (-Xmx). The required overhead depends on the application and its configuration and should be determined through testing.

Pointing JAVA applications to node local scratch directories#

On our clusters all jobs are provided with a node local scratch directory: /localscratch/$SLURM_JOB_ID. Java uses the java.io.tmpdir system property to determine the default location for temporary files. If a particular application does not allow this directory to be supplied on the command line, yet requires to write scratch files, the _JAVA_OPTIONS allows to define the scratch directory anyway:

# put this in your job submission script to specify the local scratch directory
export _JAVA_OPTIONS="-Djava.io.tmpdir=/localscratch/${SLURM_JOB_ID}/"