Met. Office Operational NWP Products -Background Information


Go to: [ Contents | Introduction | Data | Using Data | End ]

Contents

  1. Introduction
  2. The UM data
  3. Obtaining and using UM data from the BADC


Go to: [ Contents | Introduction | Data | UsingData | End ]

IMPORTANT: byte swapping of these data files may be required before they can be read on your systems. Utilities are available to perform the byte-swapping and convert these files into several other formats such as NetCDF.

Byte swapping software has been available for some time from the BADC and is described at http://badc.nerc.ac.uk/help/software/xconv/index.html .

1. Introduction

This document describes the UK Met. Office Operational NWP data products (based on the Unified Model (UM)) which have been retrieved from the Met. Office since 23 October 2000. Data files are recovered every 6 hours (0,6,12,18Z), which contain analysis fields and forecast fields out to 6 hours from these times. These files are processed by the BADC to convert then into a format more suited to the UM community (binary PP), with the analysis fields and forecasts out to 12 hours being archived at the BADC.

1.1 The UK Unified Model

The UK unified global model outputs a number of variables onto a global grid for a number of vertical levels. The initial output is onto "model" vertical levels, and from these additional processing is used to produce another set of global variables on pressure levels in the atmosphere. For the past 6 years, the output resolution was fixed (432x325, 38 model levels, 38km top height). However, in December 2005, enhancements were made to the output, so that output is now available on a 640x481 grid, at 50 model levels, top level 65km . Please see this page for more details.

The UK operational Mesoscale model uses a rotated latitude and longitude coordinate system in which the computational north pole is shifted to an actual position of 37.5 deg. N, 177.5 E. This is done to obtain fairly uniform horizontal resolution over the area of interest. The corners of the computational area are approximately in actual latitude/longitude: (60.1N, 16.6W; 60.2N, 10.7E; 46.6N, 12.7W; 46.7N, 7.1E). The grid-length is 0.15 deg. in each direction which is approximately 16.8 Km, giving 92x92 grid points running from the north - west corner.


Go to: [ Contents | Introduction | Data | UsingData | End ]

2. The UM data

2.1 Model output data fields

Model dumps are retrieved from the UKMO once each day. These consist of global and mesoscale data (on model and standard levels) generated at 0, 6, 12, and 18 Z. The output data from each model run include a variety of field types and have forecasts out to 6 hours from each step. This gives a data rate of around 1.5 Gb per day, depending on the packing method used. These files are processed to split the data into 1 file per variable/timestep. The analysis and forecast data are stored in separate directories for both mesoscale and global data. The format of the saved filenames is described in section 3.

The following table shows the most common Field codes (Note these are not the same as STASH codes).

 

Table of common pp field codes.

1
 Height

8
Pressure, p

16
 Temperature, T

28
 d(p*)/dt p*=surface pressure

39
 Coriolis parameter

40
 Omega (=dp/dt)

43
 Eta dot

56
 Westerly cpt of wind, u

57
 Southerly cpt of wind, v

61
 'x' cpt of wind stress

62
 'y' cpt of wind stress

79
 QCL- liquid cloud water

95
 Specific Humidity, q

Note that all winds are held on the U-grid. The U and V winds on Pressure levels are true Westerly and Southerly wind components but all other winds are orientated with the model grid.

2.1 Field Code to STASH Code Mapping

A text file describing the Met Office full list of stash and field codes for the UM model is also available.
This may be useful when examining NWP data held at the BADC under the "old" filenaming convention.


Go to: [ Contents | Introduction | Data | UsingData | End ]

2.2 Spatial coverage and resolution

2.2.1 Horizontal coverage

Global Data: The data points are now given on a global (640x480) longitude-latitude grid, giving resolutions of 0.56 and 0.375 deg (prior to Dec 2005 it used a 432 x 324 grid, giving 0.83 and 0.56 deg.respectively.

Mesoscale Data: The data points are given on a (146 x 181) longitude-latitude grid, giving a resolution of 0.11 deg, which is approximately 12.3 Km.

The corners of this grid are located at (353.00 368.96 11.44 -8.36), on a non-standard polar axis system (polar lat,lonitude= 37.5,177.0).

 

2.2.2 Vertical coverage

The upper level of the global model level output is now 65 km (previously 38km), and there are 50 model levels (previously 38).

The data fields are output on standard pressure levels and model (hybrid) levels, for global and mesoscale runs.

There are now 27 standard pressure levels in the output global datafiles, and 13 pressure levels for the mesoscale data.

 

UM Global Data Standard Pressure levels (mbar)

 Pressure (mbar)

 Global

Mesoscale

1000

 ˆ

 ˆ

950

 ˆ

 ˆ

925

 ˆ
 

900

 ˆ
 

850

 ˆ

 ˆ

700

 ˆ

 ˆ

600

 ˆ
 

500

 ˆ

 ˆ

400

 ˆ

 ˆ

300

 ˆ

 ˆ

250

 ˆ

 ˆ

200

 ˆ

 ˆ

150

 ˆ

 ˆ

100

 ˆ

 ˆ

70

 ˆ

 ˆ

50

 ˆ
 

30

 ˆ

 ˆ

20

 ˆ
 

15

 ˆ
 

10

 ˆ
 

7

 ˆ
 
5
 ˆ
3
 ˆ
2
 ˆ
1
 ˆ
0.5
 ˆ
0.4
 ˆ

 

 

2.3 Temporal coverage and resolution

The data held in the NWP archive can now be viewed on-line for the period 1/1/2004-30/6/2005: global data mesoscale data . We are currently continuing to fill in the gaps in the archive as quickly as possible.

From 2000-2003, the datafiles available at the BADC were extracted from raw UM dump files, which contained both analyses and forecasts. From 2004 onwards, the data were recovered from the Met Office MASS system.

Analyses are available every 6 hours for most of the fields, and corresponding forecasts up to 6 hours are current archived at the BADC.

For pre-2004 data, there may be forecasts greater than 6 hours available -please see below for details of the "old" archive. It may also be possible to retrieve UM data for dates prior to 23 October 2000, if required -please contact the BADC for further information.


Go to: [ Contents | Introduction | Data | Using Data | End ]

3. Obtaining and using UM data from the BADC

3.1 The UM data files

These data are IEEE binary files in Met. Office "PP" format, consisting of unpacked headers (mostly integer data) followed by floating point field data. There are text files available descibing the Met Office format for the PP file header, and a full list of the stash codes used.

Old Data, New Data

Up until mid 2003, data in the BADC archive were stored in what we term the "old" filenaming convention. With the Met Office move to Exeter, the archive was reasssesed, and it was decided to change the filenaming convention to identify the files by stash code, and to have a similar directory structure to that used for other model data held at the BADC, such as ECMWF.

The use of field codes has proved confusing, as field codes relate to a broad description of a parameter (such as "field code 16 = temperature field", but variables with different "stash" codes (usually unique) can have the same field code. For example, "temperature on theta levels" (stash code = 16004) and "temperature at 1.5m" (stash code = 03236), would both be stored in the same file under the old naming convention, but in separate files under the new. This new convention makes it much simpler to locate the field of interest.

We are currently recovering data from the Met Office archive from 2000-2004 to fill the gaps in our archive. This will then provide a consistent archive of NWP data using the "new" filenaming convention.

Old Data (2000-2003)

For "old" data, the directory structure is of the form year, month, day, creation time, um-type, filename, where creation time is 00, 06, 12, 18 and um-type file is : mg (model levels/global), mm (model levels/mesoscale), sg (standard levels/global), sm (standard levels/mesoscale).

The individual datafiles are in pp format, with a separate file for each paramter and timestep. They have filenames of the form:

 m  g 2000 11 13 00  p8 s00 .pp
 1  2 3 4 5 6 7 8  

1: m/s (model/standard) levels
2: g/m (global/mesoscale) data
3: year
4: month
5: day
6: time of file (00,06,12,18)
7: parameter field code (pp)
8: step(hr from time of file (6))

for example:

mg2000111300p8s00.pp : model levels, global, yr=2000, m=11, d=13, run at 00hrs, parameter=1 (pressure), step=0 (analysis).

mm2000111300p8s00.pp : model levels, mesoscale ..(as above).

sg2000111300p8s00.pp : standard levels, global ..(as above).

sm2000111300p8s00.pp : standard levels, mesoscale ..(as above).

The parameter codes are as given in the UM model documentation, and the forecast steps are up to 6 hrs only.

New Data (2004-present)

Data collected since 1/1/2004 are being held in the following way. These files now have 1 diagnostic variable per file per timestep (rather than the field code used previously -see above).

Mesoscale : lb[a,f][m,p]yyyymmddhh_STASHCODE_fh.pp

Global: ag[a,f][m,p]yyyymmddhh_STASHCODE_fh.pp

where yyyymmddhh is the year, month,day and assimilation time, STASHCODE is the STASHMASTER parameter code, fh is the forecast timestep (from the assimilation time), and pp indicates that the files are in binary "pp" format.

The directory structure has also been changed to bring it in line with the BADC ECMWF holdings. The mesoscale files and global data are now stored under:

/badc/ukmo-um/data/meso/lb/a[m,p] for the analysis fields

/badc/ukmo-um/data/meso/lb/f[m,p] for the forecast fields

/badc/ukmo-um/data/global/ag/a[m,p] for the analysis fields

/badc/ukmo-um/data/global/ag/f[m,p] for the forecast fields

Note that mesocale data is now held under "meso", rather than "mesoscale".

So for example, global analysis data on pressure levels for the variable with stashcode=16202 at 0Z on 1/6/2005 would be referenced as:
/badc/ukmo-um/data/global/ag/ap/2005/06/01/agap2005060100_16202_00.pp.

Pre-2004 data are still available in the old format under /badc/ukmo-um/data/mesocale/[sm,mm] and /badc/ukmo-um/data/global/[mg,sg], although these will be phased out as the data are archived in the newer format. We have also converted the global pre-2004 data to the new format, and these are also available under /badc/ukmo-um/data/global/ag/a[m,p]


Go to: [ Contents | Introduction | Data | Using Data | End ]

3.2 Software to read the data

Both old and "new" data can be read using the following utilities.

As mentioned above, the UM data are stored in very large binary files, so software is required to read the files and extract the fields of interest.

Xconv/convsh:

Xconv is a program designed to convert model output into a format suitable for use in various plotting packages (such as NetCDF). The package is simple to use, and has a X-windows based interface. It can be used to see what fields are contained within a data file, and to look at the data values, either directly or via a graphical plot of the data. Xconv can also be used to manipulate the input data before it is converted into chosen output format.

Convsh is a command line package which allows scripts to be written to automate various xconv tasks. Convsh uses the Tcl scripting language, plus various extensions for reading, writing and manipulating data files.

http://www.ugamp.rdg.ac.uk/~jeff/xconv/

These utilities are now available from the BADC . These also allow the user to convert these files into several other formats such as NetCDF. A subsetting utility based on Xconv/convsh is also provided.

3.3 Software to manipulate and plot the data

Ferret: http://ferret.wrc.noaa.gov/Ferret

Xconv/convsh: http://www.ugamp.rdg.ac.uk/~jeff/xconv/

Software to rotate the co-ordinates to allow for the offset pole position of the mesoscale data may also be required.

Please report any problems with this page to: BADC support.