Managing reservations

A reservation is a set of nodes that for a period of time can only be used by a single project.

Reserved nodes will not automatically be used, you need to specify which jobs should use the reserved nodes (using the --reservation option to sbatch/interactive/salloc).

Some possible use cases for reservations:

• You are going to work interactively with e.g visualization or software development at a specific time and want guaranteed access to nodes.

• A course or training event needs guaranteed access to compute nodes during a lab session.

Reservations are managed using the command nsc-boost-reservation. Run nsc-boost-reservation --help to see all command line options that are available:

Cost

The base cost of creating a reservation is 10 tokens per core hour¹. If you cancel a reservation before it has started. your project will receive a full refund of the reservation cost. If you cancel a reservation after it has started, your project will be receive a partial refund.

Special types of projects which a higher than normal need to create reservations can in some cases get a lower cost. Contact NSC Support for more information about this.

Checking if a reservation can be created (nsc-boost-reservation --check)

Example: check if it is possible to reserve two nodes for two hours at a specific time

[kronberg@sigma ~]$ nsc-boost-reservation --check --starttime=2019-11-09T18:00 --nodecnt=2 --duration=2:0:0 --account=nsc
It is possible to make this reservation!

Now run nsc-boost-reservation with the --create option instead of --check.
You also need to add the --name option to name your reservation.
To check the cost before making the reservation, add the --show-cost option.

Example: if it is not possible to create a reservation starting at the specified time, nsc-boost-reservation will suggest the earliest possible time when the reservation can start (in this case 2019-11-09 20:48:12).

[kronberg@sigma ~]$ nsc-boost-reservation --check --starttime=2019-11-09T18:00 --nodecnt=20 --duration=2:0:0 --account=nsc
It is not possible to reserve the nodes requested from 2019-11-09T18:00:00
The earliest possible time is 2019-11-09 20:48:12
Did not find enough nodes before absolute end date: Running jobs are blocking. More nodes @2019-11-09 20:48:12

Create a reservation (nsc-boost-reservation --create)

Example: check the cost of reserving two nodes for two hours at a specific time

[kronberg@sigma ~]$ nsc-boost-reservation --create --starttime=2019-11-09T18:00 --nodecnt=2 --duration=2:0:0 --account=nsc --show-cost
You did not specify a name for the reservation, generated one (bt-kronberg-27853) for you
The cost for creating the reservation is 32640.0 tokens
Project nsc has 967973.3 tokens available

Example: create the reservation

[kronberg@sigma ~]$ nsc-boost-reservation --create --starttime=2019-11-09T18:00 --nodecnt=2 --duration=2:0:0 --account=nsc --name=eveninghack
Your reservation bt-eveninghack has been created.
The cost was 32640.0 tokens.
Project nsc now has 935333.3 tokens available

Your reservation details:
ReservationName=bt-eveninghack StartTime=2019-11-09T18:00:00 EndTime=2019-11-09T20:00:00 Duration=02:00:00
   Nodes=n[1221,1237] NodeCnt=2 CoreCnt=64 Features=thin PartitionName=sigma Flags=REPLACE_DOWN
   TRES=cpu=64
   Users=(null) Accounts=nsc Licenses=(null) State=INACTIVE BurstBuffer=(null) Watts=n/a

To see existing reservations, run "sinfo -Tl" or "scontrol show reservation"

Example: reserve one node for one hour, starting now (leave out --starttime to choose “now” as the start time)

[kronberg@sigma ~]$ nsc-boost-reservation --create --name=mydevel --nodecnt=1 --duration=1:0:0 --account=nsc
No start time given, assuming you want the reservation to start in one minute (2019-11-10T10:43:10)
Your reservation bt-mydevel has been created.
The cost was 16160.0 tokens.
Project nsc now has 967680.0 tokens available
[...]

Note: it is only possible to create a reservation starting now if there are actually idle nodes available.

Note: the reservation name will always begin with “bt-“. If the name you chose does not, “bt-“ will be prepended to the reservation name. If you do not specify a name, a random one will be chosen.

Note: even if nsc-boost-reservation --check says that it’s possible to create a reservation, nsc-boost-reservation --create can occasionally fail to create it. This happens if a job started on the nodes that were available to reserve between the time the check command run and the time you tried to actually create the reservation. If this happens, run nsc-boost-reservation --check again, it will tell you when nodes will be available, and then run nsc-boost-reservation --create with the new start time.

Using a reservation

All members of the project can use the reservation.

Jobs will only use the reservation if you specifically request it (using the --reservation option to e.g sbatch or interactive).

Jobs that have requested to use the reservation will not start on nodes outside the reservation, even if idle nodes are available.

Example: start an interactive session on the reserved nodes

[kronberg@sigma ~]$ interactive -N2 -t 00:10:00 --reservation=bt-eveninghack -A nsc
salloc: Granted job allocation 681937
srun: Step created for job 681937
[kronberg@n1185 ~]$ 

Example - submit several batch jobs that will run in the reservation:

[kronberg@sigma ~]$ sbatch -N2 -t 00:10:00 --reservation=bt-eveninghack -A nsc sleep.sh 
Submitted batch job 681938
[kronberg@sigma ~]$ sbatch -N2 -t 00:10:00 --reservation=bt-eveninghack -A nsc sleep.sh 
Submitted batch job 681939
[kronberg@sigma ~]$ sbatch -N2 -t 00:10:00 --reservation=bt-eveninghack -A nsc sleep.sh 
Submitted batch job 681940

Note: if you submit more work than fits on the reserved nodes, the remaining jobs will be queued and start when there are available nodes in the reservation.

If you want to monitor only jobs that has requested the reservation use squeue --reservation=...

It is possible to submit a job to a reservation that has not yet started. The job will then not start until the reservation starts.

To view details of existing reservations, use the command sinfo -Tl (overview) or scontrol show reservation (show all reservations details).

To visually see the running jobs and upcoming reservations you can also use the graphical scheduler status display for Tetralith or Sigma.

Cancelling a reservation (nsc-boost-reservation --cancel)

We strongly encourage you to cancel a reservation that you will not be using. You can cancel a reservation before or after it has started.

When you cancel a reservation your project will receive a refund of tokens. Add the option --show-cost to see how large the refund would be.

Any member of the project can cancel any reservation belonging to the project.

Example - cancel the reservation bt-eveninghack

[kronberg@sigma ~]$ nsc-boost-reservation --cancel --name=eveninghack
Your reservation bt-eveninghack has been cancelled.
The refund was 32640.0 tokens.
Project nsc now has 967973.3 tokens available