Get started with MX at HPC

Both NSC Tetralith and LUNARC Aurora can be accessed after download and installation of ThinLinc client from Cendio. After Thinlinc login many MX softwares can be conveniently launched using the PReSTO menu. The PReSTO menu launch graphics MX software, ccp4mg/pymol/coot/chimera with OpenGL acceleration at the login node and also perform slurm scheduling for parallel applications such as XDSAPP/XDSGUI/hkl2map using a dialogue box to select project number, number of cores and allocate compute time. When the PReSTO menu is used with XDSAPP/XDSGUI/hkl2map the GUIs starts starts at the compute node after slurm scheduling by the dialogue box.

We recommend using the PReSTO menu to launch applications however equivalent alternatives using a terminal window would be:

PReSTO menu alternatives to launch graphics software in OpenGL accelerated mode using terminal window

module load CCP4
vglrun coot

module load CCP4
vglrun ccp4mg

module load Chimera
vglrun chimera

module load PyMOL
vglrun pymol

PReSTO menu alternatives to launch interactive and parallel applications XDSAPP/XDSGUI/hkl2map using terminal window

interactive --nodes=1 --exclusive -t 01:00:00 -A snic2020-5-368
module load XDSAPP
xdsapp

interactive --nodes=1 --exclusive -t 01:00:00 -A snic2020-5-368
module load XDSGUI
xdsgui

interactive --nodes=1 --exclusive -t 01:00:00 -A snic2020-5-368
module load hkl2map
hkl2map

You can find more information about using the ThinLinc desktop environment from Lunarc support pages or NSC running graphical applications.

Modules available in PReSTO

A module avail XXX shows all available versions or modules of XXX

module load PyMOL                 # load PyMOl default module
module load PyMOL/2.1.0-2-PReSTO  # load PyMOL version 2.1.0

A module load XXX or module load XXX/version command is required in sbatch scripts and prior to launching software from the terminal window.

Letter case matter when loading modules
module load CCP4            # load default CCP4 module
    ccp4i                   # launch ccp4i GUI
    ccp4i2                  # launch ccp4i2 GUI
    vglrun coot             # run coot   in OpenGL accelerated mode at NSC Tetralith
    vglrun ccp4mg           # run ccp4mg in OpenGL accelerated mode at NSC Tetralith
module load Chimera         # load Chimera default module
    vglrun chimera          # run chimera in OpenGL accelerated mode (crashes without vglrun) at NSC Tetralith
module load PyMOL           # load PyMOL default module 
    vglrun pymol            # run pymol in OpenGL accelerated mode at NSC Tetralith
module load CNS             # load CNS default module
    cns_solve               # usually part of cns sbatch script launched from command-line of login node
module load Phenix          # load Phenix default module
    phenix                  # launch Phenix GUI
module load XDS             # load xds default module
    xds_par                 # execute xds in parallel mode
    xds_par                 # usually part of XDS sbatch script launched from command-line of login node
module load generate_XDS.INP# load generate_XDS.INP default module
    generate_XDS.INP        # get XDS.INP from diffraction images or eiger master file
module load DIALS           # load standalone dials default module
    dials                   # part of dials sbatch script
module load XDSAPP          # load XDSAPP default module
    xdsapp                  # launch XDSAPP GUI
    xdsapp --cmd            # part of xdsapp sbatch script launched from command-line of login node
module load XDS-Viewer      # load XDS-Viewer default module
    xds-viewer              # launch XDS-Viewer
module load XDSSTAT         # load XDSSTAT, called by XDSGUI and XDSAPP etc.
module load ALBULA          # load ALBULA default module
    albula                  # launch ALBULA GUI
module load adxv            # load adxv default module
    adxv                    # launch adxv GUI
module load hkl2map         # load hkl2map default module
    hkl2map                 # launch hkl2map GUI
module load SHARP           # load SHARP default module, called from autoSHARP
    run_autoSHARP.sh        # usually part of autoSHARP sbatch script launched from command-line of login node
module load BUSTER          # load BUSTER default module
    refine                  # usually part of BUSTER sbatch script launched from command-line of login node
module load XDSME           # load XDSME default module
    xdsme                   # usually part of XDSME sbatch script launched from command-line of login node
module load autoPROC        # load autoPROC default module
    process                 # usually part of autoPROC sbatch script launched from command-line of login node
module load USF             # load all softwares from Uppsala software factory
    moleman                 # run moleman from command-line
    moleman2                # run moleman2 from command-line

HPC terminology

HPC terminology regarding nodes, cores, processors and tasks taken from the LUNARC Aurora pages on the subject.

Term Explanation Tetralith Aurora

node

A physical computer

1892

180

processor

a multi-core processor housing many processing elements

2 per node

2 per node

socket

plug where processor is placed, synonym for the processor

2 per node

2 per node

core

individual processing element

32 per node

20 per node

task

software process with own data & instructions forking multiple threads

specified in sbatch script

thread

An instruction stream sharing data with other threads from the task

specified in sbatch script

Table 1. Terminology regarding resources as defined in the HPC community

HPC vs XDS terminology

Term XDS terminology

tasks

JOBS (i.e. MAXIMUM_NUMBER_OF_JOBS)

threads

PROCESSORS (i.e. MAXIMUM_NUMBER_OF_PROCESSORS)

Table 2. HPC vs XDS terminology

Useful HPC commands

HPC command Consequence

interactive -N 1 --exclusive -t 00:30:00 -A snic2020-5-368

Get 30 min terminal window at compute node for project

interactive --nodes=1 --exclusive -t 00:30:00 -A snic2020-5-36

Get 30 min terminal window at compute node for project

exit

leave terminal window, save compute time on project

snicquota

check your diskspace at /home and /proj/xray folder

snicquota -a

check diskspace for all users in /proj/xray

squeue -u x_user

Check my jobs in running or in the SLURM queue

top

see all jobs running on current node

top -U username

see my jobs running on current node

scancel JOBID

Kill my job with JOBID

module load XDSAPP

Load XDSAPP and dependencies CCP4, PHENIX, XDS, XDSTAT...

module avail

What modules are there

module purge

unload all modules

Table 3. Basic HPC commands

Tetralith and Aurora differences

Tetralith vs Aurora outcome

jobsh n1024

Access a node when in use at NSC Tetralith

ssh au118

Access a node when in use at Lunarc Aurora

Table 4. Command-line commands differing between Lunarc Aurora and NSC Tetralith

SBATCH and checking the queue

Running sbatch scripts is the most efficient way of using HPC compute time since once the job is finished, the clock counting compute time is stopped. Every sbatch script require a maximum time for the job to finish by #SBATCH -t 00:30:00 before the job can be scheduled into the queue. To check status of jobs sumitted by sbatch use squeue -u username and obtain

Figure 1. Output of squeue -u username. The first column gives the job ID, the second the partition (or queue) where the job was submitted, the third the name of the job (specified by the user in the submission script) and the fourth the owner of the job. The fifth is the status of the job (R=running, PD=pending, CA=cancelled, CF=configuring, CG=completing, CD=completed, F=failed). The sixth column gives the elapsed time for each particular job. Finally, there are the number of nodes requested and the nodelist where the job is running (or the cause that it is not running).

Now it is possible to access the compute nodes and check job status in more detail using top or top -U username. This is done in two different ways at NSC Triolith and LUNARC Aurora:

  • At LUNARC Aurora use: ssh au118
  • At NSC Tetralith use: jobsh n1024

Once your terminal window is at the compute node you can check status by top

Figure 2. Result of top given at compute node. Using top the status of the job is indicated by (D=uninterruptible sleep, R=running, S=sleeping, T=traced or stopped, Z=zombie)

or by top -U username

Figure 3. Result of top -U username given at compute node

The interactive command can use the same parameters as sbatch below.

SBATCH script line Consequence

#!/bin/sh

Use sh to interpret the script

#SBATCH -t 0:30:00

Run the sh script for maximum 30 min

#SBATCH --nodes=2 --exclusive

Allocate two full nodes for this sh script

#SBATCH -A snic2017-1-XXX

Count compute time on project snic2017-1-XXX

#SBATCH --mail-type=ALL

Send email when job start and stops

#SBATCH --mail-user=name.surname@lu.se

Send email to name.surname@lu.se

Table 5. SBATCH script lines. The interactive command is using the same terminology, however usually given in a single row at the login node as interactive --nodes=2 --exclusive -t 00:30:00 -A snic2017-1-XXX

Useful LINUX commands

  • rsync -rvplt ./data username@aurora.lunarc.lu.se:/lunarc/nobackup/users/username/. data directory copied to lunarc
  • scp file.pdb username@aurora.lunarc.lu.se:/lunarc/nobackup/users/username/. single file.pdb copied to lunarc

User Area

User support

Guides, documentation and FAQ.

Getting access

Applying for projects and login accounts.

System status

Everything OK!

No reported problems

Self-service

SUPR
NSC Express