Cluster Cosmology

h1. How to run jobs on the euclides nodes

Use slurm to submit jobs to the euclides nodes (node1-8), ssh login access to those nodes will be restricted in the near future.

h2. alexandria

*Please do not use alexandria as a compute node* - it's hardware is different from the nodes. It hosts our file server and other services that are important to us. 

You should use alexandria to

- transfer files

- compile your code

- submit jobs to the nodes

If you need to debug, please start an interactive job to one of the nodes using slurm. For instructions see below.

h2. euclides nodes

Job submission to the euclides nodes is handled by the slurm jobmanager (see http://slurm.schedmd.com and https://computing.llnl.gov/linux/slurm/). 

*Important: In order to run jobs, you need to be added to the slurm accounting system - please contact Kerstin*

All slurm commands listed below have very helpful man pages (e.g. man slurm, man squeue, ...). 

If you are already familiar with another jobmanager the following information may be helpful to you http://slurm.schedmd.com/rosetta.pdf‎.

h3. Scheduling of Jobs

<some more information will appear here as we figure out how to organize the scheduling>

h3. Running an interactive job with slurm

To run an interactive job with slurm (for example for debug purposes)

<pre>

srun -u bash -i

</pre>

As soon as a slot is open, slurm will log you in to an interactive session on one of the nodes.

h3. Running a simple once core batch job with slurm

* To see what queues are available to you (called partitions in slurm), run:

<pre>

sinfo

</pre>

* To run slurm, create a myjob.slurm containing the following information:

<pre>

#!/bin/bash

#SBATCH --output=slurm.out

#SBATCH --error=slurm.err

#SBATCH --mail-user <put your email address here>

#SBATCH --mail-type=BEGIN

#SBATCH -p normal

/bin/hostname

</pre>

* To submit a batch job use:

<pre>

sbatch myjob.slurm

</pre>

* To see the status of you job, use 

<pre>

squeue

</pre>

* For some more information on your job use

<pre>

scontrol show job <jobid>

</pre>

the <jobid> you can get from using squeue.

h3. Batch script for running a multi-core job

To run a 4 core job you can use

<pre>

#!/bin/bash

#SBATCH --output=slurm.out

#SBATCH --error=slurm.err

#SBATCH --mail-user <put your email address here>

#SBATCH --mail-type=BEGIN

#SBATCH -n 4

mpirun <executable>

</pre>