Berzelius Storage is intended for storage of actively used data associated with ongoing computing projects on Berzelius.1 Storage is allocated to Berzelius projects according to the Berzelius storage allocation policy, and is limited both by volume (bytes), and number of files stored.2
Being a limited resource, storage must be used efficiently and in a way that does not cause problems for NSC staff or other projects. This includes using suitable file formats, compressing data when appropriate, packaging data in archive formats (e.g tar, zip) when appropriate and also includes how applications perform I/O (e.g not doing unnecessary I/O or I/O patterns that cause problems for other users)
Projects that continue under another name can keep their storage directory name
(/proj/projname
), but the amount of data that can be stored is determined by
the continuation project’s allocation.
Data that remains in a project directory after the project has ended (and is not directly followed by a continuation project) will3 by this policy be handled as follows
As mentioned, we will attempt to send reminder emails before data deletion happens, but we do not guarantee it (accidents and mistakes happen), securing scientific results is the responsibility of the researchers at all times.
“Warm” data storage is available on other systems, e.g Swestore for longer term data storage. ↩
There are two reasons to limit the number of files stored by a project. 1: certain operations, like checking/repairing the file system, and starting it after certain types of crashes, takes time proportional to the number of files in it. 2: every file, even empty ones, consumes a certain amount of storage space for metadata (filename, permissions, timestamps, …), which is not counted towards the normal quota. The files limit is shown in SUPR. We will typically be generous when asked to raise this limit, it acts mostly as a tripwire to alert us to when a project starts storing data in a problematic way (tens of millions of small files). ↩
Currently, we delete the data manually at some point 60 or more days after the project ends. If you notice that your data is missing but there’s more than 60 days since the project ended, please get in touch with NSC ASAP, if you are really lucky we have not yet deleted the data. ↩
Guides, documentation and FAQ.
Applying for projects and login accounts.