Supported Rainfall File Formats
- Mukta Sapkota
The following provides a summary of the rainfall file formats that are supported by MUSICX.
BoM 6 minute Rainfall File (*.txt)
A fixed format file type, typically supplied by the Australian Bureau of Meteorology foe 6 minute pluviograph data. The file contains two header lines:
Line 1 contains the station identified and an integer (1);
Line 2 contains the station number and name.
Data lines start with the station number, followed by date (yyyymmdd), then 240 values of 6 minute data for that day. All values are on a fixed spacing. A sample file is provided below:
9741 1 9741 2 ALBANY AIRPORT 9741 1965 1 1-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999 9741 1965 1 2-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999 9741 1965 1 2-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999 9741 1965 1 3-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999 9741 1965 1 4-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999 9741 1965 1 5-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999.0-9999 .... 9741 1965 413 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.1 1.2 0.0 0.0 0.0 0.0 9741 1965 414-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888 9741 1965 415 0.7 4.4 4.1 1.6 4.3 4.3 0.2 2.7 2.9 0.4 0.0 0.0 0.0 2.0 1.9 0.6 0.0 0.0 0.0 0.7 1.4 9741 1965 416 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.9 1.9 0.0 0.0 9741 1965 419 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 9741 1965 420-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888 9741 1965 421-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888.0-8888 9741 1965 422 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 |
CDT Comma Separated (.cdt)
A comma delimited column time series format file that contains regular (periodic) time series data. The file commonly has no header line, but can support a single line header specifying “Date, Time series 1” if required.
There are two columns of data, with the first column being a date string, followed by the time series value. The date string should be specified using the ISO date-time string format (e.g. 2010-01-24 00:06). The second column contains the rainfall value. The columns are separated by a comma.
Note: We strongly recommend using zero padding when defining dates, e.g. use “01/01/1996” for 1st January 1996, not 1/1/1996.
Below is an example of a suitably formatted 6 minute CDT file:
Date,Time series 1 2000-01-01,0.727869398 2000-01-02,0.569338308 2000-01-03,1.934807009 2000-01-04,0.944831487 2000-01-05,3.547859312 |
CSV Comma Separated (.csv)
A comma separated variable file is an ASCII text file that contains regular time series data (i.e., missing periods of data cannot be accommodated). There are two columns of data, with the first column being a time-date string (e.g. 01/01/2003 12:06:00), followed by the time series value. The two values are separated by a comma and the time should be in 24-hour format.
The file may have a header line with a title for each column. The first column header is usually “Date-Time” followed by a relevant data column header (e.g. rainfall_mm).
The following is an example of a correctly formatted CSV file:
Date, Timeseries 1 2000-01-01,0.727869398 2000-01-02,0.569338308 2000-01-03,1.934807009 2000-01-04,0.944831487 2000-01-05,3.547859312 2000-01-06,0.94145698 |
Hydrolog (.dtf)
A fixed format file type that is output by the Hydrolog software (software used for the storage of time series data). The file has a 15-line header followed by time series data values
Note that data stored in WIKSI can be output in various formats. Therefore, care should be taken to adhere to the file format set out below.
The lines of information contained within the header are as follows. Words in bold appear literally in the file:
LINE | PURPOSE |
|---|---|
1 | File header (e.g., Hydrolog3 File Export) |
2 | User :, followed by a user name (e.g., JohnP) |
3 | Database: , followed by the database the data was extracted from |
4 | Created on : followed by the file creation date |
5 | Date Requested : followed by the start and end date/time of the time series (e.g., 21/12/2001 09:00:00 to 30/01/2002 08:59:59) |
6 | Date Format:, followed by the date format used for the time series (e.g., dd/mm/yyyy) |
7 | Quality Codes:, followed by quality codes identifier (e.g., Hydrolog4) |
8 | Blank line |
9 | Station ID:, followed by station identification number/string |
10 | Station Name:, followed by the station name |
11 | Parameter Name:, followed by the name of the measured data type (e.g., Rainfall) |
12 | Parameter Unit:, followed by the units of measurement (e.g., mm) |
13 | Archive Type:, followed by a description of the archive type |
14 | Blank line |
15 | Header line, typically, Date/Time ,Interpolated Data,Quality Code |
After the header, the data is organised in three columns that are separated by spaces:
Column 1: Data and time in DD/MM/YYYY HH:MM:SS format (note the single space between date and time fields);
Column 2: Rainfall value in mm;
Column 3: Quality code;
An example of a correctly formatted .dtf file is provided below:
Tarsier Daily Time Series (.tts)
The Tarsier daily time series format file is an ASCII text file containing daily time series data. It has a 21-line header followed by daily data values.
The lines of information contained within the header are as follows. Words in bold appear literally in the file:
LINE | PURPOSE |
|---|---|
1 | The Tarsier version number header |
2 | Reference to author of Tarsier modelling framework |
3 | File path and name |
4 | Name of software used to create the file |
5 | Date and time file was created |
6 | Tarsier timer series data class (eg TTimeSeriesData) |
7 | File version number |
8 | Number of header lines (set to 1) |
9 |
|
10 | Number of daily data entries in the file |
11 | Xlabel is always Date/Time for time series data |
12 | Y1Label Y1 fixed field, doesn’t change |
13 | Y2Label Y2 fixed field, doesn’t change |
14 | Units followed by Data units |
15 | Format followed by format information (eg 1) |
16 | Easting followed by grid position east in metres |
17 | Northing followed by grid position north in metres |
18 | Latitude followed by the latitude of the site in decimal degrees |
19 | Longitude followed by the longitude of the site in decimal degrees |
20 | Elevation followed by the elevation of the site in metres |
21 | Header character (usually an asterisk; ASCII 42, ASCII hex 2A) |
After the header, the data is organised into four columns that are separated by spaces:
Column 1: Year;
Column 2: Julian day of that year;
Column 3: Data value;
Column 4: Data quality cod - A period/dot (ASCII value 46, ASCII hex value 2E) means that the data is ok/ present. A dash (ASCII value 45, ASCII hex value 2D) means that the data is missing.
The following provides an example of TTS file:
WISKI All (.all) and WISKI CSV (.csv)
The WISKI ALL file is an ASCII text file that contains time series data. WISKI stands for Water Information System KISTERS and is a common format for the storage of hydrologic time series.
Note that data stored in WIKSI can be output in various formats. Therefore, care should be taken to adhere to the file format set out below.
The file has a 21-line header followed by daily data values. The lines of information contained within the header are as follows. Words in bold appear literally in the file:
LINE | PURPOSE |
|---|---|
1 | Station name, followed by a comma and then the station name (e.g., Bedford, Summerhouse Hill no 2) |
2 | Station number, followed by the station number as a string surrounded with double quotes (e.g., "174053") |
3 | External number, followed by an integer station number (e.g., 174053) |
4 | River, followed by the river name |
5 | Operator, followed by an operator description |
6 | NRG, followed by a the NGR identifier (e.g., TL) |
7 | Easting, followed by the easting (i.e., X coordinate) of the station |
8 | Northing, followed by the northing (i.e., Y coordinate) of the station |
9 | Parameter-name, followed by the data type name (e.g., Rainfall) |
10 | Parameter Type, followed by the data type code (e.g., RE) |
11 | Time series name, followed by a time series identifier (e.g., 174053.RE.15.Total) |
12 | Time series unit, followed by a unit identifier (e.g., mm) |
13 | Time level, followed by a resolution description (e.g., High-resolution) |
14 | Time series type, followed by a time series type identifier (e.g., Total) |
15 | Equidistant time series, followed by a boolean value (i.e., yes or no) |
16 | Time series value distance, followed by the time-step of the time series (e.g., 15 Minute(s)) |
17 | Time series quality, followed by a quality description (e.g., Production) |
18 | Time series measuring method, followed by a description of the measurement type (e.g., Recorder) |
19 | Period of record in file, followed by the start and end date/time of the time series (e.g., 01/08/1991 15:15:00 to 20/06/2010 00:00:00) |
20 | Quality code description, followed by a list of quality code abbreviations |
21 | Header file for time series (e.g., Date Time,RF [mm],Quality flag) |
After the header, the data is organised in three columns that are separated by comma:
Column 1: Data and time in DD/MM/YYYY HH:MM:SS format (note the single space between date and time fields);
Column 2: Rainfall value in mm;
Column 3: Quality code;
An example of a WIKSI All file is provided below:
Met Office CSV (.csv)
A comma delimited, fixed format file type. It contains a header followed by the time series data. Note that rainfall is always read from the fifth column of the file.
An example of a Met Office CSV file is provided below: