Data Format and Transfer Specifications

Using Community Conventions

updated 4/25/03

(PDF VERSION – Coming Soon!)

 

APE participants need only provide the subset of APE diagnostics from which all others can be derived.  This minimum set reduces the required diagnostics to the GT, SH, TR, ML, MF and PF tables (see APE diagnostic specifications).   The diagnostics listed in GA, SZ, MZ, TE and VB will be derived later from the minimum set.

 

We are exploiting the efforts of many who have worked to develop data standards for the climate research community.   While a little investment is required to become familiar with the conventions, it is important to realize the benefit: this community activity is working towards a common means of data exchange that will be applicable for many other WCRP projects (e.g., AMIP and CMIP).

 

The data standards described below presume that APE models have recti-linear longitude-latitude grids or are interpolated to such a grid. Gaussian grids are of course acceptable, unless the number of longitude points varies with latitude.   To ensure that APE data can be efficiently managed and results rapidly analyzed, PCMDI will only be able to accept data that meets the following specifications:

 

  • Data must be provided in netCDF and compliant with minimum specifications defined by the CF convention

Note:  Participants need not follow all CF specifications, only those listed below.

 

  • CF Convention:

Dimension Information

Dimension names:  

    •     "time" for the time dimension
    •     "plev" (data on pressure levels) or “lev” (data on model surfaces – table PF)
    •     "longitude" or “lon” for the longitude dimension
    •     "latitude" or “lat” for the latitude dimension


Dimension units:

    • No Time dimension for annual mean fields
    • time: 'days since 0000-1-1-0' for daily (mean or instantaneous) data
    • plev: ‘Pa’
    • longitude: 'degrees_east'
    • latitude: 'degrees_north'

Note: The year (0000), month (1) and day (1) have been arbitrarily chosen for the APE configuration. Integrations should begin 0000-01-01, meaning if you follow the recommendation for a 6 month spin-up the data submitted to PCMDI will begin 0000-07-01. For the 6hourly data (table TR), fractional days should be used (e.g., 0000-07-1.25) and the reference time should include 0-hours (e.g., "days since 0000-01-01-00") to anchor the diurnal cycle.  

 

Dimension attributes:

    • time: Calendar  -   ‘NoLeapCalendar’ or ‘Calendar360’
    • latitude: latitude_bounds (see example netCDF output below)

 

 

Data on model surfaces (table PF only):

    • CF accommodates transform coefficients (the A’s and B’s)
    • Relevant references in CF documentation

1.    Dimensionless vertical coordinate (Section 4.3.2)

2.    Dimensionless coordinate types  (Appendix C)

3.    Sample file (ncdump output) given below


Grid ordering and orientation:  (time[0, N], level[ near-surface level, top level], latitude[Southernmost latitude, Northernmost latitude.] , longitude[0,eastern most point]).    For lesser ranking fields, preserve the ordering, e.g., SZ fields (single level zonal means): (time, latitude).

 

 

Variable attributes:

    • Each variable (identified by the attribute name) should have the associated attributes ape_name and units.
    • These required attributes, separated by a ‘\’, are provided in the attached files (below).   The contents of each file (e.g., gt-cf-info.txt) include: 1) the APE diagnostic (e.g., GT01), 2) the APE variable name (e.g., ‘gt_sw_toai’), 3) the ape_name (e.g., ‘toa_incoming_shortwave_flux’) and, 4) units (e.g., W m-2):

 

 

Only the variables listed in these files need to be provided.  The remainder of the diagnostics in the APE diagnostic specifications will be derived from this minimum set.  The majority of the ape_names are consistent with the CF "standard names."

 

Sample CF compliant netCDF output (using the ncdump command):

 

 

Global attributes:

Conventions: ‘CF-1.0’

Source: Model Version as defined by modeler

 

  • File Structure

Data files should be organized according to Analysis/Data (e.g., GT, SH, TR; see APE diagnostic specifications) and experiment.  File names should include information about the modelling group, the data type, and the experiment with the following template for file names:  modelvintage_data_experiment.nc ,  where the ‘.nc’ extension implies a netCDF file.  All the variables of the same data type (e.g., GT, SH) should be stored in the same file, giving 6 files/experiment, giving 6x8 = 48 files per model.  Example file names:

NCAR-03a_GT_control.nc               (Global time series)

NCAR-03a_SH_qobs.nc                 (Single-level 2-D means)

NCAR-03a_TR_control5n.nc           (Transients)

NCAR-03a_MF_3keq.nc                 (Multiple-level zonal mean fluxes)

NCAR-03a_ML_control.nc              (Multiple-level 3-D means)

 

·        The 8 experiments are identified by: ‘control’, ‘peaked’, ‘flat’, ‘qobs’, ‘control5n’,’1keq’,’3keq’,’3kw1’

·        The 6 data types save for each experiment are: GT, SH, TR, ML, MF, PF

 

It is recommended that the AMIP convention for ‘model id’ be used to identify the model.  This simply designates the group running the model (e.g., ‘ncar’) and the year it was run (e.g., ‘03a’).  The letter (‘a’) appended to the year is a place holder in case more than one run is submitted in a given year.  Note, at PCMDI we more completely refer to a ‘model designation’ that is unique for each ‘model vintage’. See for example: http://www-pcmdi.llnl.gov/amip/STATUS/incoming.html.

 

Before processing all of your experiments, we reccomend that you prepare your control run according to these standards, and submit it to PCMDI for verification.

Contact Peter Gleckler (pgleckler@llnl.gov) if you have any questions.

[   APE Home   |   Proposal   |   Design   |   Diagnostics   |   Data formats   |   ASL papers   |   Partici pants   ]

Last modified: April 25 2003