Running MX software in high performance compute environment

We recommend using the PReSTO menu to launch applications however equivalent alternatives using a terminal window would be:

PReSTO menu alternatives to launch graphics software using terminal window

module load CCP4
coot

module load CCP4
ccp4mg

module load Chimera
chimera

module load ChimeraX
chimerax

module load PyMOL
pymol

How to launch interactive and parallel applications XDSAPP3/XDSGUI/hkl2map using terminal window

interactive --nodes=1 --exclusive -t 01:00:00 -A naiss2023-22-811
module load XDSAPP3
xdsapp

interactive --nodes=1 --exclusive -t 01:00:00 -A naiss2023-22-811
module load XDSGUI
xdsgui

interactive --nodes=1 --exclusive -t 01:00:00 -A naiss2023-22-811
module load hkl2map
hkl2map

You can find more information about using the ThinLinc desktop environment from Lunarc support pages or NSC running graphical applications.

Modules available in PReSTO

A module avail XXX shows all available versions or modules of XXX

module load PyMOL                 # load PyMOl default module
module load PyMOL/2.5.0-1-PReSTO  # load PyMOL version 2.5.0

A module load XXX or module load XXX/version command is required in sbatch scripts and prior to launching software from the terminal window.

Letter case matter when loading modules
module load Phenix          # load Phenix default module
    phenix                  # launch Phenix GUI
module load XDS             # load xds default module
    xds_par                 # execute xds in parallel mode
    xds_par                 # usually part of XDS sbatch script launched from command-line
module load generate_XDS.INP# load generate_XDS.INP default module
    generate_XDS.INP        # get XDS.INP from diffraction images or eiger master file
module load DIALS           # load standalone dials default module
    dials                   # part of dials sbatch script
module load XDSAPP          # load XDSAPP default module
    xdsapp                  # launch XDSAPP GUI
    xdsapp --cmd            # part of xdsapp sbatch script launched from command-line
module load XDS-Viewer      # load XDS-Viewer default module
    xds-viewer              # launch XDS-Viewer
module load ALBULA          # load ALBULA default module
    albula                  # launch ALBULA GUI
module load adxv            # load adxv default module
    adxv                    # launch adxv GUI
module load hkl2map         # load hkl2map default module
    hkl2map                 # launch hkl2map GUI
module load SHARP           # load SHARP default module, called from autoSHARP
    run_autoSHARP.sh        # usually part of autoSHARP sbatch script launched from command-line
module load BUSTER          # load BUSTER default module
    refine                  # usually part of BUSTER sbatch script launched from command-line
module load XDSME           # load XDSME default module
    xdsme                   # usually part of XDSME sbatch script launched from command-line 
module load autoPROC        # load autoPROC default module
    process                 # usually part of autoPROC sbatch script launched from command-line
module load CNS             # load CNS default module
    cns_solve               # usually part of cns sbatch script launched from command-line
module load USF             # load all softwares from Uppsala software factory
    moleman                 # run moleman from command-line
    moleman2                # run moleman2 from command-line

HPC terminology

HPC terminology regarding nodes, cores, processors and tasks taken from the LUNARC Cosmos pages on the subject.

Term Explanation Tetralith Cosmos

node

A physical computer

1892

182

processor

a multi-core processor housing many processing elements

2 per node

2 per node

socket

plug where processor is placed, synonym for the processor

2 per node

2 per node

core

individual processing element

32 per node

48 per node

task

software process with own data & instructions forking multiple threads

specified in sbatch script

thread

An instruction stream sharing data with other threads from the task

specified in sbatch script

Table 1. Terminology regarding resources as defined in the HPC community

HPC vs XDS terminology

Term XDS terminology

tasks

JOBS (i.e. MAXIMUM_NUMBER_OF_JOBS)

threads

PROCESSORS (i.e. MAXIMUM_NUMBER_OF_PROCESSORS)

Table 2. HPC vs XDS terminology

Useful HPC commands

HPC command Consequence

interactive -N 1 --exclusive -t 00:30:00 -A naiss2023-22-811

Get 30 min terminal window at compute node for project

exit

leave terminal window, save compute time on project

storagequota

check your diskspace at /home and /proj/xray folder

storagequota -a

check diskspace for all users in /proj/xray

squeue -u x_user

Check my jobs in running or in the SLURM queue

top

see all jobs running on current node

top -U username

see my jobs running on current node

scancel JOBID

Kill my job with JOBID

module load XDSAPP

Load XDSAPP and dependencies CCP4, PHENIX, XDS, XDSTAT...

module avail xxx

What xxx modules are there?

module purge

unload all modules

Table 3. Basic HPC commands

Tetralith and Cosmos differences

Tetralith vs Cosmos outcome

jobsh n1024

Access a node when in use at NSC Tetralith

ssh au118

Access a node when in use at Lunarc Cosmos

Table 4. Command-line commands differing between LUNARC Cosmos and NSC Tetralith

SBATCH and checking the queue

Running sbatch scripts is the most efficient way of using HPC compute time since once the job is finished, the clock counting compute time is stopped. Every sbatch script require a maximum time for the job to finish by #SBATCH -t 00:30:00 before the job can be scheduled into the queue. To check status of jobs sumitted by sbatch use squeue -u username and obtain

Figure 1. Output of squeue -u username. The first column gives the job ID, the second the partition (or queue) where the job was submitted, the third the name of the job (specified by the user in the submission script) and the fourth the owner of the job. The fifth is the status of the job (R=running, PD=pending, CA=cancelled, CF=configuring, CG=completing, CD=completed, F=failed). The sixth column gives the elapsed time for each particular job. Finally, there are the number of nodes requested and the nodelist where the job is running (or the cause that it is not running).

Now it is possible to access the compute nodes and check job status in more detail using top or top -U username. This is done in two different ways at NSC Triolith and LUNARC Cosmos:

  • At LUNARC Cosmos use: ssh au118
  • At NSC Tetralith use: jobsh n1024

Once your terminal window is at the compute node you can check status by top

Figure 2. Result of top given at compute node. Using top the status of the job is indicated by (D=uninterruptible sleep, R=running, S=sleeping, T=traced or stopped, Z=zombie)

or by top -U username

Figure 3. Result of top -U username given at compute node

The interactive command can use the same parameters as sbatch below.

SBATCH script line Consequence

#!/bin/sh

Use sh to interpret the script

#SBATCH -t 0:30:00

Run the sh script for maximum 30 min

#SBATCH --nodes=2 --exclusive

Allocate two full nodes for this sh script

#SBATCH -A naiss2023-22-811

Count compute time on project naiss2023-22-811

#SBATCH --mail-type=ALL

Send email when job start and stops

#SBATCH --mail-user=name.surname@lu.se

Send email to name.surname@lu.se

Table 5. SBATCH script lines. The interactive command is using the same terminology, however usually given in a single row at the login node as interactive --nodes=2 --exclusive -t 00:30:00 -A naiss2023-22-811

Useful LINUX commands

  • rsync -rvplt ./data username@aurora.lunarc.lu.se:/lunarc/nobackup/users/username/. data directory copied to lunarc
  • scp file.pdb username@aurora.lunarc.lu.se:/lunarc/nobackup/users/username/. single file.pdb copied to lunarc

User Area

User support

Guides, documentation and FAQ.

Getting access

Applying for projects and login accounts.

System status

Everything OK!

No reported problems

Self-service

SUPR
NSC Express