MARS

Meteorological Archival and Retrieval System (MARS) is an archiving system for meteorological data developed at ECMWF and used at SMHI to archive both operational and research data. MARS accepts meteorological data in GRIB1, GRIB2 format and in principle BUFR and ODB. For further reference, see the ECMWF MARS user documentation.

The SMHI MARS archive is accessible from all nodes on Bi after loading its module (see below).

Support

For support please use mars-support@nsc.liu.se.

Before using MARS

First make sure you have all MARS components loaded by

module load mars

The latest version of the MARS client uses grib_api 1.12.3 to read/validate GRIB messages and emoslib 392 to manipulate/transform them (interpolate, crop, etc.). You also need to load the python module (currently 2.7.6-smhi1) which is used in various MARS subcomponents. If you did not load the grib_api, emos and python modules, the mars module will do it for you, making sure that the correct version of these libraries are used.

If it is the first time you are using MARS, you will need to generate a key (SSH key) to send to the MARS administrator when you apply for an account. This is described in the following section.

Getting an account

The first time you use one of the MARS components, a pair of SSH keys will be generated. The pair of keys is used for authentication on the MARS servers. Note that these are not the common SSH keys you may have in your ~/.ssh directory on Bi, but a separate pair used solely to contact MARS and stored in ~/.mars directory.

During the first attempt to contact MARS, you will get on-screen instructions on how to proceed to get an account on the MARS servers. Once you have an account, the next step is to create an “experiment id” (expid) to tag your data as being your data if you are planning to archive data on the research MARS server. If you only want to retrieve data you don’t need an expid.

Note that if you have an account on the old mars1 and mars2 servers, you will have to go through the registration process again (new SSH keys will not be created but you will still have to contact us to get access to marsre and marsop.

Using MARS Operational production

On the server for operational data, marsop, one can only list or retrieve data, but not archive anything.

Available Operational Model

Currently, marsop contains the following datasets (soon MetCoOp AROME 2,5km as well under expver a25a):

EXPVER Description
C11A Hirlam C11 11km operational run
C22A Hirlam C22 22km operational run
E11A Hirlam E11 11km operational run
E05A Hirlam E05 5km operational run
G05A Hirlam G05 5km operational run
H01A Hiromb BS01 forecast operational run
H03A Hiromb NS03 short forecast operational run
HL3A Hiromb NS03 long forecast operational run

Retrieving

For general information on how to retrieve data from MARS read the ECMWF MARS user documentation.

A typical request to retrieve initial 2m temperatures at synoptic time 00/06/12/18 +00H for last summer (june-july-august 2013) would be:

RETRIEVE,
	CLASS    = OP,
	STREAM   = OPER,
	EXPVER   = C11A,
	MODEL    = HIRLAM,
	TYPE     = FC,
	DATE     = 20130601/TO/20130831/BY/1,
	TIME     = 00/06/12/18,
	STEP     = 0,
	LEVTYPE  = HL,
	LEVELIST = 2,
	PARAM    = 11.1,   
	TARGET   = "T2m.grb"

Save this request in a file, for instance T2m.retrieve, and submit it using the mars command:

[sm_sebvi@krypton ~]$ mars T2m.retrieve 
tun  - INFO   - 20131106.095047.317730 - Using SSH tunnel configuration within /tmp/mars.tunnel.3369.UYWh8A
mars - INFO   - 20131106.095047 - Welcome to MARS with grib_api
mars - INFO   - 20131106.095047 - MARS_HOME=/software/apps/mars_client/client/2.6/gcc446
mars - INFO   - 20131106.095047 - MARS Client version: 20110905
mars - INFO   - 20131106.095047 - EMOSLIB version: 392
mars - INFO   - 20131106.095047 - Using grib_api version 1.11.0
	RETRIEVE,
		CLASS    = OP,
		STREAM   = OPER,
		EXPVER   = C11A,
		MODEL    = HIRLAM,
		TYPE     = FC,
		DATE     = 20130601/TO/20130831/BY/1,
		TIME     = 00/06/12/18,
		STEP     = 0,
		LEVTYPE  = HL,
		LEVELIST = 2,
		PARAM    = 11.1,   
		TARGET   = "T2m.grb"

mars - INFO   - 20131106.095047 - Processing request 1

RETRIEVE,
    CLASS      = OP,
    TYPE       = FC,
    STREAM     = OPER,
    EXPVER     = c11a,
    MODEL      = HIRLAM,
    LEVTYPE    = HL,
    LEVELIST   = 2,
    PARAM      = 11.1,
    DATE       = 20130601/20130602/20130603/20130604/20130605/20130606/20130607/20130608/20130609/20130610/20130611/20130612/20130613/20130614/20130615/20130616/20130617/20130618/20130619/20130620/20130621/20130622/20130623/20130624/20130625/20130626/20130627/20130628/20130629/20130630/20130701/20130702/20130703/20130704/20130705/20130706/20130707/20130708/20130709/20130710/20130711/20130712/20130713/20130714/20130715/20130716/20130717/20130718/20130719/20130720/20130721/20130722/20130723/20130724/20130725/20130726/20130727/20130728/20130729/20130730/20130731/20130801/20130802/20130803/20130804/20130805/20130806/20130807/20130808/20130809/20130810/20130811/20130812/20130813/20130814/20130815/20130816/20130817/20130818/20130819/20130820/20130821/20130822/20130823/20130824/20130825/20130826/20130827/20130828/20130829/20130830/20130831,
    TIME       = 0000/0600/1200/1800,
    STEP       = 0,
    DOMAIN     = G,
    TARGET     = "T2m.grb",
    DATABASE   = marsop

mars - INFO   - 20131106.095047 - Requesting 368 fields
mars - INFO   - 20131106.095047 - Calling mars on 'localhost', callback on 4207
mars - INFO   - 20131106.095047 - Callback at address 127.0.0.1, port 4207
mars - INFO   - 20131106.095048 - Mars client is on localhost (127.0.0.1) 4207
mars - INFO   - 20131106.095048 - Mars server is on localhost (127.0.0.1) 54156
mars - INFO   - 20131106.095048 - Server task is 562 [marsop]
mars - INFO   - 20131106.095049 - Request cost: 368 fields, 111.882 Mbytes online, nodes: <missing-node-name> [marsop]
mars - INFO   - 20131106.095050 - Transfering 117316914 bytes
mars - INFO   - 20131106.095053 - Request time:  wall: 6 sec 
mars - INFO   - 20131106.095053 -   Processing in marsop: wall: 3 sec 
mars - INFO   - 20131106.095053 -   Visiting marsop: wall: 5 sec 
mars - INFO   - 20131106.095053 -   Writing to target file: 111.89 Mbyte(s) in 2 sec  [70.15 Mbyte/sec]
sm_sebvi@krypton ~]$

Listing

It is possible to querry the MARS servers using a “LIST” request. For instance, one can ask if marsop contains 10 meters wind components u and v for the model C11, June 2013:

LIST,
    CLASS     = OP,
    MODEL     = HIRLAM,
    TYPE      = FC,
    STREAM    = OPER,
    EXPVER    = C11A,
    DATE      = 20130601/TO/20130630,
    TIME      = ALL,
    STEP      = ALL,
    LEVTYPE   = HL,
    LEVELIST  = 10,
    PARAM     = 33.1/34.1,
    TARGET    = "list.out"

Save this request in a file, for instance list.request and use submit it with the mars command:

[sm_sebvi@krypton ~]$ mars list.request 
tun  - INFO   - 20131106.095726.589504 - Using SSH tunnel configuration within /tmp/mars.tunnel.3369.UYWh8A
mars - INFO   - 20131106.095726 - Welcome to MARS with grib_api
mars - INFO   - 20131106.095726 - MARS_HOME=/software/apps/mars_client/client/2.6/gcc446
mars - INFO   - 20131106.095726 - MARS Client version: 20110905
mars - INFO   - 20131106.095726 - EMOSLIB version: 392
mars - INFO   - 20131106.095726 - Using grib_api version 1.11.0
LIST,
    CLASS     = OP,
    MODEL     = HIRLAM,
    TYPE      = FC,
    STREAM    = OPER,
    EXPVER    = C11A,
    DATE      = 20130601/TO/20130630,
    TIME      = ALL,
    STEP      = ALL,
    LEVTYPE   = HL,
    LEVELIST  = 10,
    PARAM     = 33.1/34.1,
    TARGET    = "list.out"

mars - INFO   - 20131106.095726 - Processing request 1

LIST,
    CLASS      = OP,
    TYPE       = FC,
    STREAM     = OPER,
    MODEL      = HIRLAM,
    EXPVER     = c11a,
    LEVTYPE    = HL,
    LEVELIST   = 10,
    DUPLICATES = KEEP,
    PARAM      = 33.1/34.1,
    DATE       = 20130601/20130602/20130603/20130604/20130605/20130606/20130607/20130608/20130609/20130610/20130611/20130612/20130613/20130614/20130615/20130616/20130617/20130618/20130619/20130620/20130621/20130622/20130623/20130624/20130625/20130626/20130627/20130628/20130629/20130630,
    TIME       = ALL,
    STEP       = ALL,
    HIDE       = missing,
    TARGET     = "list.out",
    OUTPUT     = TABLE,
    DATABASE   = marsop

mars - INFO   - 20131106.095726 - Calling mars on 'localhost', callback on 4207
mars - INFO   - 20131106.095726 - Callback at address 127.0.0.1, port 4207
mars - INFO   - 20131106.095726 - Mars client is on localhost (127.0.0.1) 4207
mars - INFO   - 20131106.095726 - Mars server is on localhost (127.0.0.1) 54550
mars - INFO   - 20131106.095726 - Server task is 562 [marsop]
mars - WARN   - 20131106.095744 - Closing connection with 1 byte outstanding
mars - INFO   - 20131106.095744 - Request performed on database 'marsop'
mars - INFO   - 20131106.095744 - Request time:  wall: 18 sec 
mars - INFO   - 20131106.095744 -   Processing in marsop: wall: 18 sec 
mars - INFO   - 20131106.095744 - No errors reported
[sm_sebvi@krypton ~]$

The list requests tend to output quite a lot of cryptic informations. It is usually a good idea to include the “HIDE” and “OUTPUT” keywords in the request:

LIST,
   CLASS     = OP,
   MODEL     = HIRLAM,
   TYPE      = FC,
   STREAM    = OPER,
   EXPVER    = C11A,
   DATE      = 20130601/TO/20130630,
   TIME      = ALL,
   STEP      = ALL,
   LEVTYPE   = HL,
   LEVELIST  = 10,
   PARAM     = 33.1/34.1,
   HIDE      = GRAND-TOTAL/CUBES/DATE_INTERVAL,
   OUTPUT    = TREE,
   TARGET    = "list.out"

the content of “list.out” indicates that the data is available:

[sm_sebvi@krypton ~]$ more list.out 
class=op,date=2013-06-01,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-02,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-03,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-04,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-05,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-06,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-07,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-08,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-09,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-10,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-11,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-12,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-13,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-14,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-15,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-16,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-17,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-18,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-19,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-20,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-21,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-22,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-23,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-24,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-25,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-26,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-27,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-28,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-29,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
class=op,date=2013-06-30,expver=c11a,levtype=hl,model=hirlam,month=201306,stream=oper,type=fc,year=2013
[sm_sebvi@krypton ~]$

Using MARS Research

On the server containing research data, marsre, users are allowed to archive their personal experiments, provided it is in GRIB format. We cover here, how to get one or more experiments IDs to “tag” your data as yours and how to preprocess your data so MARS understands it.

Getting an experiment ID

Since MARS does not store any ownership details within its database, an experiment id (expid), specified in the request through the “EXPVER” keyword, is used to distinguish between the data owned by different users. This experiment id is exactly 4 characters long and may consist of any combination of the digits 0-9 and the ASCII character set a-z (or A-Z, MARS does not make a distinction between upper and lower case). The experiment id’s are stored in a separate database, called expid database and are used to genereate MARS permissions on the fly to protect your data.

There are few things especially noteworthy regarding the experiment id’s:

  • An experiment id is only unique to a certain MARS class (research or operational), therefore the MARS class must always be specified as a mandatory argument when creating an id. In practice, you will never create an id on the operational server, marsop, for safety reasons. So the MARS class will always be “re” for all regular users. This will change in the future if we decide to open a third MARS server, for special usage or datasets.
  • An allocated id is stored on a MARS server together with the user (you, called “owner”), with optionally other users with archiving permissions (called “writers”, useful for collaborative project) and with a supplied (up to 43 character long) description of the id.
  • You can create as many ids as you like, you are not limited to only one. Ideally, you should create a new id for each project or dataset you are working on.
  • There are no restrictions/permissions regarding retrieving/listing your data. If you have an account on a MARS server, you can retrieve data from whatever id on that server. Your data will be retrievable by anyone having a MARS account.
  • An allocated id can not be updated/removed by any other than the owner of the id. A “writer” can not modify an id.
  • Removing an id is a two-stage process. First you “de-allocate” the id, then a MARS administrator physically remove the data and the id from the database. In between, you have the possibility to change your mind and “restore” that id (and get access back to your data). You can restore the id as long as you can see it listed preceded by the mention “DELETED”.
  • If you are tempted to re-use one of your id because you no longer need its content, don’t!!! In three years time, you will forget what belongs to which project in that id, and it is more complicated for a MARS administrator to selectively delete some of the data than wiping away the entire content of the id. If you really want to re-use an id, contact us for a fast delete.

Here are few examples on how to manage your expids:

  1. List configured MARS classes:
    expid –list–classes
  2. use the research id database in interactive mode:
    expid –class=re –interactive
  3. Allocate a new experiment id for the server marsre (research):
    expid –class=re –allocate -id A007 –description=”This is James Bond’s id”
  4. Update a previously allocated id to add writers:
    expid –class=re –update –id=A007 –writers=m,q,moneypenny
  5. Delete (de-allocate) a previously allocated id:
    expid –class=re –delete –id=A007
  6. Restore a previously deleted id (valid if not processed by the MARS administrator yet):
    expid –class=re –restore –id A007
  7. List all allocated id:s on marsop (operational):
    expid –class=op –list
  8. List all allocated id:s on marsre by the user bond:
    expid –class=re –list –owner=bond

The expid command is available to you once you loaded the MARS module.

Preprocessing

Before archiving in marsre, you need to prepare your data so MARS understands it and decides where to keep it and how to organize it. To do so, you need to encode within the local section of your GRIB messages (octet 40-80 in section 1 for GRIB edition 1) a set of keys that MARS understands (CLASS, STREAM, TYPE, MODEL and EXPVER) and sometimes some extra ones (for the “MATCH” model for instance or for the stream “EXPR”). Here is a table with the MARS keywords that should be encoded in the GRIB local section:

EXPVER id as generated by expid, see previous section
CLASS always “re” on marsre (research)
STREAM type of data, usually “OPER” for marsop or “EXPR” for marsre
TYPE type of integration, typically “AN” for analysis or “FC” for forecast
MODEL type of model: HIRLAM, HIROMB, AROME, SURFEX, etc…
EXPOFFSET only needed if STREAM=EXPR, usually 0<
TIMEREPRES only used with MATCH model
SORT only used with MATCH model
LANDTYPE only used with MATCH model
AEROSOLBINNUMBER only used with MATCH model

Most of the keywords are self-explained.
On marsre, you will always use CLASS=RE. Depending on the type of data you are archiving, you will have to choose the right STREAM. It is recommended that by default you use STREAM=EXPR, which is the intended default STREAM on marsre. This is typically for a one-time experiment over a limited range of time, typically one model run. When you use that STREAM, you should use it combined with the EXPOFFSET keyword. The offset allow you to store the same experiment a second, a third, a fourth, N times while changing few settings in your model. This way you get all your related experiments under the same expid, with expoffset=0 or 1 or 2, etc. This is ideal while you test several values of a parameter in your model.

Adding local sections

The easiest way to add local sections in your GRIB files is to use grib_api:

grib_set -ssetLocalDefinition=82,localDefinitionNumber=82,class=RE,expver=MYID,type=FC,model=HIRLAM,stream=EXPR,expoffset=0 input.grb output.grb

This command reads GRIB messages from the file input.grb and writes them in output.grb, after including the local definition 82 of local centre 82. In details, “setLocalDefinition=82” refers to SMHI (with WMO CCCC code being “eswi”), “localDefinitionNumber=82” refers specifically to local definition 82 (within the range of local section defined by SMHI), and the other keywords are pretty self-explaining.

In practice, “setLocalDefinition” is usually set to 82 (at least for now) and “localDefinitionNumber” can be either 82 (all models but MATCH) or 83 (MATCH model). In the future, we will probably create new local sections needed to describe other type of data.

Note that the whole procedure assumes that your GRIB messages are encoded with centre=82 exclusively (maybe in the future, metcoop and harmonie centre will be added) and not centre=96 (old coding used at SMHI for hirlam data) or centre=98 (ecmwf). To set “centre” to 82 before including the local section, use:

grib_set -s centre=82 grib_without_centre_set_to_82.grb grib_with_centre_set_to_82.grb

However, you need to be very careful with what you are doing. For instance, suppose you have a GRIB message from ECMWF (centre=98) containing temperature (coded with ECMWF definitions indicatorOfParameter=130, table2Version=128). If you simply set centre=82 without changing indicatorOfParameter and table2Version to SMHI definitions for temperature, you create a very confusing and dangerous situation because parameter 130 of table 128 at SMHI is undefined. In the worst case scenario, the parameter could be defined but to something completely different!

Organizing GRIB messages for efficient archiving

MARS sorts and organizes data logically for efficient retrieving and compact storage. A typical model output is usually organized in files per steps per synoptic hours because the model usually writes data to disk as soon as a specific step is finished. However, common retrieving patterns are “all temperatures for all the steps of a specific day or month”, or “pressure levels over a period of time”, or “precipitations for a specific season”, not “give me all the GRIB messages of a specific step from a specific synoptic hours”. For this reason, MARS groups your data in a different way than you have it in files. Previously, this was sorted out by a tool called “himar.x” but now it is replaced by a set of grib_api tools.

It is maybe complicated at first, but in practice, it is rather simple. MARS uses a tree-branches-leaves scheme to organize GRIB files. The MARS keywords can either be the nodes of the branches or axis within the leaves.

If a MARS keyword is used as a node of the MARS tree, that keyword must have a unique value during archiving KEYWORD=VALUE (but you have total freedom during listing, staging or retrieving). MARS keywords that are always used as node are CLASS, STREAM, EXPVER, MODEL, TYPE, and LEVTYPE:

    CLASS    = op,
    STREAM   = oper,
    EXPVER   = a007,
    MODEL    = hirlam,
    TYPE     = fc,
    LEVTYPE  = MSL,

If a MARS keyword is used within a MARS leaf (also called hypercube, because values of the leaf keywords become axis of a n-dimensional array or cube), several values of that keyword can be listed, separated by “/”:

    TIME     = 00/06/12/18,   # could be shortened with 00/TO/18/BY/6
    STEP     = 00/TO/48/BY/3, # this is equivalent to /00/03/06/.../42/45/48
    LEVELIST = 01/TO/60/BY/1, # this equivalent to /01/02/.../59/60
    PARAM    = 11.1/91.1,     # no compact way to specify parameters

MARS keywords that are always hypercube axis are TIME, STEP, PARAM, and LEVELIST.

Currently, only the DATE keyword needs special attention. MARS uses the DATE keyword to extract the meta keywords YEAR, MONTH and DAY. If data is compacted by DAY, you can only archive one DATE at a time. If data is compacted by MONTH, you can archive several dates belonging to the same month. If data is compacted by YEAR, you can archive several dates belonging to the same year.

    DATE     = 20130903,             # only one date: September 3rd 2013
    ...
    DATE     = 20130901/TO/20130930, # one month: September 2013
    ...
    DATE     = 20130101/TO/20131231, # one year: 2013

How do you know if you can group data per DAY, MONTH or YEAR? Well there is no easy answer. A new tool will soon be available to do this for you.

If you decide to do things yourself, the easiest way to sort/group your GRIB messages is to use the grib_api tool called grib_filter:

grib_filter filter_rules in_grib_file.grb in_grib_file.grb ....

grib_filter applies to each GRIB messages a set of rules described in an input file “filter_rules”. Typical filter files can be found in /software/apps/mars_client/filter, covering the most common user cases. If you are dealing with unusual or more complex case, have a look at the official grib_api documentation or contact the mars support.

Generating MARS archive requests

Once you have your GRIB files ready, you need to prepare your MARS requests. The easiest way to achieve this is to use the “grib2request” tool (provided by the mars module). grib2request does not actually produce a valid MARS request stricto sensu but will give you very good hints on what it should look like.

[sm_sebvi@krypton ~]$ grib2request 
Usage: grib2request -1 -f <file>
          -1 : print a single request
          -f : file to process
[sm_sebvi@krypton ~]$ grib2request -1 -f e4od_19951012_FC_ML.grb
GRIB,
    DOMAIN     = G,
    LEVTYPE    = ML,
    LEVELIST   = 1/2/3/4/5/6/7/8/9/10/11/12/13/14/15/16/17/18/19/20/21/22/23/24/25/26/27/28/29/30/31/32/33/34/35/36/37/38/39/40/41/42/43/44/45/46/47/48/49/50/51/52/53/54/55/56/57/58/59/60,
    DATE       = 19951012,
    TIME       = 0000/0600/1200/1800,
    STEP       = 3/6/9/12/18/24/30/36/42/48,
    PARAM      = 11.1/33.1/34.1/51.1/58.1/71.1/76.1/200.1/61.1/115.1/116.1,
    CLASS      = RE,
    TYPE       = FC,
    STREAM     = REAN,
    EXPVER     = e4od,
    MODEL      = HIRLAM

grib2request - INFO   - 20131106.092323 - Decoded 21720 GRIB messages
grib2request - INFO   - 20131106.092323 - Request time:  wall: 35 sec  cpu: 11 sec

The “-1” option is almost always needed. the “-f” is used to specify which file to process. What is left for you to do is to change “GRIB” for “ARCHIVE”, and add the “SOURCE” keyword . Save this (remove of course the last two lines starting with “grib2request - INFO”) in a file called for instance “archive.request”

Using a compact tool to add local definitions, sort GRIB messages and create archive requests.

not available yet.

Archiving

Once you have preprocessed your data, archiving is a pretty simple task:

mars archive.request

Removing data

It is not possible for a user to remove data from the data himself.

Retrieving EURO4M data

The Euro4M Reanalysis datatset is currently stored on marsre. You can use the following keyword to retrieve it:

DATABASE MARSRE
EXPVER E4OD (HIRLAM 11km) OR E4DD (MESAN or HIRLAM high resolution)</TT></TD></TR>
CLASS RE
STREAM REAN
TYPE AN or FC
MODEL HIRLAM OR MESAN
LEVTYPE HL,ML,PL,etc.
LEVELIST integer, depends on LEVTYPE keyword
PARAM 1.1,11.1,etc.
DATE from 19790101 to 20140228
TIME 00, 06, 12 or 18
STEP 0 with TYPE=AN, 3,6,9,etc. with TYPE=FC
TARGET filename to write your data in.

Have a look at keywords tables at the bottom of this page if you wonder what is HL, ML, PL, etc. Don’t hesitate to contact us at mars-support@nsc.liu.se if you have any questions/requests regarding Euro4M dataset.

MARS from non-NSC sites (MISU, etc.)

For MARS to work the username supplied from the client must match that recorded on the server, meaning that from non-NSC sites you will have to supply your NSC username on the command line when the mars client is invoked.

Differences between MARS at ECMWF and SMHI/NSC

There are some syntax differences between ECMWF and SMHI/NSC on how a MARS request is written. These differences reflects the adaptation of the MARS system to SMHI needs in the form of new keywords or extra values for the existing keywords.

The main difference is probably the introduction of the MODEL keyword which does not exist at ECMWF but which is mandatory here at SMHI.

Leveltypes

At ECMWF all surface data are archived as LEVTYPE=SFC. At SMHI, we don’t group surface data into one single leveltype but keep a one-to-one relationship between GRIB code for leveltype and their labels.

APPENDIX

User groups and corresponding mars servers

Server Database Identification User group
Marsop marsop CLASS=op Operational data
Marsre marsre CLASS=re Research data

MARS@NSC GRIB tables

CLASS

LABEL GRIB CODE Description
OP 80 SMHI operations
RE 81 SMHI research

STREAM

LABEL GRIB CODE Description
OPER 1025 Operational deterministic model daily archive
MNTH 1043 Monthly means
MODA 1071 Monthly means of daily means
MDFA 1075 Monthly means of daily forecast accumulations
DAME 4000 Daily means
SEME 4001 Seasonal means
SECY 4002 Seasonal daily cycle
EXPR 8200 Research Experiments
REAN 8299 Reanalysis Datasets (EURO4M)

MODEL

LABEL GRIB CODE Description
HIRLAM 30 NWP LAM
RCA 31 Climate model
HIROMB 40 Ocean model
MATCH 50 Atmospheric dispersion model
ALARO 20 ALARO part of Harmonie
SURFEX 21 SURFEX part of Harmonie
AROME 22 AROME part of Harmonie
IFS 10 Global NWP
MESAN 50 Mesoscale analysis system
SWAN 60 Wave model

LEVTYPE

ECMWF SMHI GRIB1 CODE Description
SFC SURF 1 SURFace level
SFC TATM 8 Top of ATMosphere level
PL PL 100 Pressure Level
SFC MSL 102 Mean Sea Level
  HMSL 103 Height above Mean Sea Level
HL HL 05 Height (above ground) Level
  LHL 106 Layer between Height Level
ML ML 09 Model Level
  LDL 112 Layer between Depth below Land surface
PT PT 13 Potential Temperature
PV PV 17 Potential Vorticity
DP DP 60 DePth (below sea) level
  ND 91 North Direction tile
  NED 192 North East Direction tile
  ED 93 East Direction tile
  SED 194 South East Direction tile
  SD 95 South Direction tile
  SWD 196 South West Direction tile
  WD 97 West Direction tile
  NWD 198 North West Direction tile
SFC ATM 200 entire ATMosphere level

TYPE

LABEL GRIB CODE Meaning
AN 02 Analysis
3V 05 3DVAR analysis
4V 06 4DVAR analysis
FC 09 Forecast
SI 21 Climate simulation

GRIB for MATCH Model

For further information about MATCH conversions please contact Lennart Robertson at SMHI.

– Page author: Sebastien Villaume


User Area

User support

Guides, documentation and FAQ.

Getting access

Applying for projects and login accounts.

System status

Everything OK!

No reported problems

Self-service

SUPR
NSC Express