Note: This is documentation for version 5.12 of Source. For a different version of Source, select the relevant space by using the Spaces menu in the toolbar above
Calibration analysis - SRG
Overview
Description and rationale
Source includes a number of optimisation techniques and statistical measures for automated model calibration and to assist modellers with the evaluation of the quality of calibration. These are mainly intended for calibrating catchment rainfall-runoff models, but are also applicable when calibrating river system models (e.g. see Lerat et al., 2013). The available automatic optimisation algorithms are:
- Shuffled complex evolution
- Uniform random sampling
- Rosenbrock method
Modellers have the option of selecting one optimisation technique or combinations two optimisation techniques (in series).
Automated calibration requires the use of an objective function to direct the optimisation process. The Source calibration tool implements single objective optimisation, which reduces the comparison between the observed and modelled data during the calibration period to a single number to be optimised (multiple objective optimisation is also available, see Multi-objective optimisation - Insight - SRGÂ for information).
Source implements five different basic types of objective function:
- Nash Sutcliffe Coefficient of Efficiency (NSE)Â
- Flow duration (specifically, the NSE of the flow duration)
- Absolute bias
- Bias penalty
- Square-root daily, exceedance and bias
The NSE can be applied to daily or monthly data, and the NSE and flow duration objectives can be applied to data that has been transformed by taking the logarithm. Source also allows the user to create some composite objective functions, of which there are two types:
- Combinations of the individual objective functions listed above. For example, the objective for calibrating streamflow at a gauging site could be a combination of the NSE and bias penalty.
- Combinations of the objectives for different model outputs. For example, a model could be calibrated using a weighted combination of the objective functions values at two or more different gauging sites.
Scale
Typically, the optimisation techniques and statistical measures are used to compare observed and estimated data at a point, such as streamflow data at a gauging station. Both the optimisation techniques and statistical measures can be applied on a daily or monthly basis.
Scientific Provenance
The statistical measures used in Source are well established. Â They are described in statistics textbooks, hydrology textbooks and papers such as Aitken (1973) and Nash and Sutcliffe (1970).
Overview information on the four optimisation techniques in Source is available in Vaze et al. (2011). Publications on the shuffled complex evolution method include papers by Duan et al. (1992) and Sorooshian et al. (1993). Publications on the Rosenbrock method include the paper by Rosenbrock (1960).
Version
Source version 3.8.8.
Dependencies
Requires observed data suitable for comparison of results from model calibration runs.
Availability
Provided with Source.
Implementation
Background
The optimisation techniques and statistical measures of calibration performance used in Source are well established and are not described in detail here. Guidance on model calibration is available in many publications, including various eWater Best Modelling Practice Guidelines (Black et al., 2011; Vaze et al., 2011; Black and Podger, 2012; and Lerat, 2012).
The choice of an appropriate objective function for calibration depends on the intended application of the model. Different objective functions are designed with the intention of emphasizing the fit of modelled flow to different aspects of the observed hydrograph (Vaze et al., 2011). The objective functions available in the Source Calibration Wizard are listed in Table 1, including useful references for further information.Â
Table 1. List of Source calibration objective functions.
Objective Function Name | Description | Reference |
---|---|---|
NSE Daily | Maximise the NSE of daily flows | Vaze et al. (2011), Section 6 |
NSE Monthly | Maximise the NSE of monthly flows | Vaze et al. (2011), Section 6 |
NSE Log Daily | Maximise the NSE of the logarithm of daily flows | |
Minimise Absolute Bias | Minimise the Absolute value of the relative bias | Vaze et al. (2011), Section 6 |
NSE Daily & Bias Penalty | Maximise the NSE of daily flows and bias penalty | Vaze et al. (2011), Section 6 |
NSE Log Daily & Bias Penalty | Maximise the NSE of the logarithm of daily flows and bias penalty | |
NSE Monthly & Bias Penalty | Maximise the NSE of monthly flows and bias penalty | Vaze et al. (2011), Section 6 |
NSE Daily & Flow Duration | Maximise the NSE of daily flows and the NSE of the flow duration | Vaze et al. (2011), Section 6 |
NSE Daily & Log Flow Duration | Maximise the NSE of daily flows and the NSE of the flow duration of log flows | Vaze et al. (2011), Section 6 |
Square-root Daily, Exceedance and Bias | Minimise a combination of the bias, daily Flows and daily exceedance (flow duration) curve | Lerat et al., 2013 |
Implementation Details
The Bivariate Statistics SRG User Guide entry provides general information on the objective function equations and their interpretation. Implementation details that are specific to the Source Calibration Wizard are described below.
Missing Data
It is common for observed time series of hydrological processes to contain missing values. Also, the observed and modelled time series may have different start and end dates. The Source calibration tool calculates the objective function values using only data from those time steps for which both observed and modelled data is available. In other words, the calibration objective function are calculated using observed and modelled data has been filtered to include only:
- data from within the calibration period, and
- data for time steps with complete data pairs.
Monthly Flows
The NSE Monthly objective function uses monthly streamflow values. These are calculated as follows:
- If the model is run on a daily time step, monthly flows are calculated by summing the observed and modelled daily flow values. The NSE calculation ignores observed and modelled data for all months where there are one or more days of missing data in the observed flow series.Â
- If the model is run on a monthly time step, then the monthly values are unchanged.
Data
Input data
Details on data to be input by the modeller are provided in the Source User Guide. Requirements for data series inputs to the various objective functions are included in the descriptions of each objective function, above.
Parameters or settings
Modellers have the option of selecting one optimisation technique, two optimisation techniques (in series), or manual optimisation. Modellers can also select which objective function they wish to use. The other parameters the modeller can input are described in Table 2.Â
Table 2. Objective function parameters. Refer to the Bivariate Statistics SRG User Guide entry for further information on their implementation.
Objective Function | Parameter | Parameter Description | Units | Default | Range |
---|---|---|---|---|---|
NSE Daily & Flow Duration | a | Weight on NSE in the combined objective | Dimensionless | 0.5 | 0 ≤ α ≤ 1 |
NSE Daily & Log Flow Duration | a | Weight on NSE in the combined objective | Dimensionless | 0.5 | 0 ≤ α ≤ 1 |
Output data
Outputs include results of the evaluation of the selected objective function and other calibration performance statistics.Â
Reference list
Aitken, A.P. (1973). Assessing systematic errors in rainfall-runoff models. J. Hydrol, 20, 121–136.
Black, D.C. and Podger, G.M. (2012). Guidelines for modelling water sharing rules in eWater Source: towards best practice model application. eWater Cooperative Research Centre, Canberra, Australia. July. ISBN: 978-1-921543-74-6. Â Available via: www.ewater.com.au.
Black, D.C., Wallbrink, P.J., Jordan, P.W., Waters, D., Carroll, C., and Blackmore, J.M. (2011). Guidelines for water management modelling: towards best practice model application. eWater Cooperative Research Centre, Canberra, Australia. September. ISBN: 978-1-921543-46-3. Â Available via: www.ewater.com.au.
Duan, Q., Sorooshian, S. and Gupta, V. (1992). Effective and Efficient global optimization for conceptual rainfall-runoff models. Water Resources Research, 28(4), 1015-1031.
Lerat, J. (2012). Towards the adoption of uncertainty assessment in water resources models: the eWater Source uncertainty guideline. Proceedings of the 34th Hydrology and Water Resources Symposium, 19-22 November 2012, Sydney, NSW.Â
Lerat, J., Egan, C. A., Kim, S., Gooda, M., Loy, A., Shao, Q., and Petheram, C. (2013). Calibration of river models for the Flinders and Gilbert catchments. A technical report to the Australian Government from the CSIRO Flinders and Gilbert Agricultural Resource Assessment, part of the North Queensland Irrigated Agriculture Strategy. CSIRO Water for a Healthy Country and Sustainable Agriculture flagships, Australia.
Nash, J.E. and Sutcliffe, J.V. (1970). River flow forecasting through conceptual models, I, A discussion of principles. J. Hydrol, 10, 282–290.
Rosenbrock, H.H. (1960). An automated method of finding the greatest of least value of a function. The Computer Journal, 3, 303-307.
Sorooshian, S., Duan, Q. and Gupta, V. (1993). Calibration of rainfall-runoff models: application of global optimization to the Sacramento Soil Moisture Accounting Model. Water Resources Research, 29(4), 1185-1194.
Vaze, J., Jordan, P., Beecham, R., Frost, A., Summerell, G. (2011). Guidelines for rainfall-runoff modelling: Towards best practice model application. eWater Cooperative Research Centre, Canberra, ACT. ISBN 978-1-921543-51-7. Available via www.ewater.com.au.