The NSC Centre Storage system provides shared file storage for the Tetralith and Sigma systems.
It is intended for short- and medium-term storage during an active project using Tetralith or Sigma.
For information on how to use the system, please see our support pages
The storage system consists of three IBM ESS GL6 building blocks and one IBM ESS GL6S building block.
The system occupies three 19" racks and consists of 8 servers and 24 disk enclosures. In total there are 1546 spinning hard disks. 1044 of the hard disks have a size of 4 TB each and the remaining 502 hard disks have a size of 10 TB each. The system also has a small number of NVRAM devices and SSDs which act as a cache to speed up small writes.
The total disk space that is usable for storing files is approximately 6200 TiB. The difference between "raw" space on the disks and the usable space in the file system is mostly due to:
The storage system is connected to Tetralith and Sigma using two Intel Omni-Path 100 Gbits/s links per server. In practice, the hard disks will often be the bottleneck for I/O, and the maximum sustained aggregated transfer speed (when writing or reading from many compute nodes simultaneously) is around 60 GiB per second.
From a single thread/core on a single Tetralith compute node you can expect to read or write up to around 1 GiB per second (as long as the disk system is not overloaded by other jobs). On login and analysis nodes this figure will be higher, around 3.5 GiB/s.
The power consumption is around 24 kW, or around 3.9 Watt per TiB of usable space.
The current NSC Centre Storage system was put into operation in October 2014. Before it was connected to Tetralith and Sigma in 2018, Centre Storage's servers and network interface was upgraded. In March 2020, a fourth building block (IBM ESS GL6S) was added, increasing the usable space from 2800 TiB to 6200 TiB.
File data data is protected by 8+2 Reed-Solomon code, i.e 8 data blocks require 2 parity blocks to be stored on disk. Meta data (file system structure, directories, contents of small files) is protected by 3-Way replication.↩