Page Comparison

The Calibration Wizard is used to calibrate the streamflow characteristics of a Source model. It can be used to calibrate rainfall runoff, constituent and link routing models and is primarily intended for use in unregulated systems. Much of the functionality relies on characteristics of catchments scenarios - specifically the presence of a sub-catchment map.

In summary, the steps involved in undertaking calibration analysis are:

• Launch Source, open your project, and then open the scenario you wish to calibrate;
• Start the wizard and work through its steps (see Calibration Wizard below);
• Click Run on the Simulation toolbar to perform a calibration run; and
• Examine the results.

Characteristics

The calibration wizard offers a lot of flexibility. It supports:

• Calibrating to single or multiple observation gauges;
• Multiple gauges in different parts of the catchment (parallel) or nested (i.e. upstream/downstream);
• Ability to calibrate modelled flow at points or links (downstream flow), and the point can be node or outlet of catchment;
• Ability to calibrate modelled constituents at points or links, and the point can be node or  (outlet of ) catchment;
• Equal weighting between gauges or different weightings (e.g. by area, length of observed record, or quality of observed record);
• Automatic calibration using a mathematical optimiser and an objective function, or manual calibration using objectives functions and visual inspection;
• Compound objective functions including the ability to manipulate weighting between objective components;
• Ability to calibrate multiple gauges for the same time period or different time periods;
• Ability to group parameters across the catchment or keep them independent, with the ability to choose this for each parameter;
• Ability to calibrate rainfall runoff models or link flow routing models or to calibrate them at the same time;
• Ability to calibrate the model with options for different Scenario Input Sets;
• Ability to calibrate any model parameters with Scenario Input Sets Calibration property;
• Ability to specify relationships between meta-parameters; and
• Ability to calibrate the model parallelly.

The Calibration Wizard is organised as a step-by-step process, which guides you through configuring the model for calibration, and a Calibration Runner, which handles the actual calibration, either in an automated fashion using an optimiser, or manually, with the ability to directly specify parameter values.

Calibration Wizard

To start the wizard, choose Flow Calibration Analysis from the Select Analysis Type popup menu on the Simulation toolbar and then click Configure, which is also on the Simulation toolbar. The wizard comprises of four steps:

• Calibration targets -define model elements ( e.g. catchment, node or link)  and associate them with the observed data loaded in the model , and select the objective function(s) that you want the optimiser to use to assess how well it fits (Figure 1);
• Period definition - define the time period(s) for your calibration. When you have more than one observation data set, you can define a distinct calibration period for each gauge. You also define a warm up period, where the overall model simulation starts at some point in time before the first calibration period commences (Figure 2, Figure 3);
• Metaparameter definition - define parameters that will be modified during the calibration to improve the model fit to the observed data. Metaparameters are distinct from the actual parameters in the model. They can correspond directly to model parameters, or multiple model parameters can be grouped together to match a single metaparameter. Usually some combination of these approaches is used to reduce the number of parameters being calibrated, while retaining distinct parameters where it is expected to improve the fit (Figure 4); and
• Select optimisation function - choose and configure the optimisation strategy, including the option of using manual calibration in lieu of optimisation (Figure 11).

Configure Calibration Points ( previous Select elements to record )

In the first wizard step, you define the point(s) or link(s) in the model where you will be calibrating to observed data (Figure 1). You can select (the outlet of) catchments, nodes or the downstream end of links, or miscellaneous if applicable. For each of these points you associate observed data and optionally provide a site weighting. You also define your calibration objectives at this step, including optionally weighting different components. You can set up the wizard to calibrate downstream from the link ,or at a node, or at the outlet from a catchment. Similarly, you can calibrate the modeled constituents.

You can select either a node or link on which to load observed data using either the Project Hierarchy or clicking the element in the geographic editor panel. The steps to load time series at a Gauge for which you have observed data are similar for a link:

• Click the gauge node in the geographic editor panel so that it becomes selected;
• Click observed flow and select Data Source radio button;
• Choose the time-series of  its associated observed data from Data Sources frame; and
• Click OK.

The same method is used to load time series for links as well.

Figure 1. Calibration Wizard (Configure Calibration Points)

At the beginning, the interface of Step 1 in Flow Calibration Wizard (i.e.  the central Windows of Figure 1) only has two components (i.e. Add New Model Element button and Input set drop down list):

Add New Model Element button is used to add a modelled element (node or link or catchment) where the observed data was loaded in Data Sources of the Source model and calibration can be performed. Clicking on this button (action 1) will add one frame of Model Element to the interface.

The user can have multiple frames of Model Element. In another word, the user can select multiple model elements (e.g. nodes or links) for the calibration at same time.

Input set drop down list allows the user to select an available Input Set for the calibration (action 2).

In each added Model Element Frame:

Model Element | Select… can open Modelled Element Selector windows by clicking on Select… (action 3). The user can navigate Modelled Element Selector windows to select the required calibration element and parameter by clicking on the triangle symbol (action 4). For example, the action 4 in Figure 1 is to select the modelled downstream flow at gauge 410024 to calibrate.

Delete button in each frame of Model Element can remove that frame from the interface. It will also remove the element, which is defined by that frame, from the calibration.

• Minimise Absolute Bias
• NSE Daily & Flow Duration
• NSE Daily & Log Flow Duration
• NSE Daily
• NSE Monthly
• NSE Daily & Bias Penalty
• NSE Monthly & Bias Penalty
• NSE Log Daily
• NSE Log Daily & Bias Penalty
• Square-root Daily, Exceedance, and Bias (previously called Sum of Daily Flows, Daily Exceedance (Flow Duration) Curve and Bias) (SDEB)

In most cases, model calibration seeks to maximize the selected objective function. The exceptions are the Absolute Bias and SDEB, which are minimized.

The general Objectives in Source  are described in (https://wiki.ewater.org.au/display/SD510/Calibration+analysis+-+SRG). The term and algorithm used in those objective function such as NSE, bias can refer to following links:

• Nash-Sutcliffe Efficiency (NSE)
• NSE of Log Data (NSE Log)
• Relative Bias
• Bias Penalty
• Pearson's Correlation Coefficient
• NSE of Flow Duration (Flow Duration)
• NSE of Flow Duration of Log Data (Log Flow Duration)

Observed Data Source | Select… can open Observed Data Selector interface by clicking on Select… (action 6). The Observed Data Selector interface allows the user to select the observed data corresponding to the selected element and parameter. The observed data need to  be loaded in the Data Sources already.

Calibration Weighting box allows you to define an alternative weighting between the objective functions defined for the elements, where you have multiple elements such as gauges. The default is 1 for each gauge, so each gauge would contribute equally to the overall objective function. The use of Calibration Weighting is optional, and the meaning of the weights can be defined by you. You might use this in a situation where for instance, there are 20 years of data at one gauge and 10 years of data at another, then apply a weighting of 2 to the first gauge and 1 to the second gauge. Alternatively, you might place higher weighting on observation points that represent more of the catchment area.

NSE Weighting box can be used to change the weighting between the components of composite objective functions. The value should be between 0 and 1 and is applied to the "first" component of the compound objective - i.e. in NSE with log flow duration, the weighting you specify will apply to a "regular" Nash Sutcliffe efficiency on the observed and predicted data (1-Weighting) and will apply to the NSE on the logged flow duration curves.

When you have finished loading time series, click Next.

Define calibration period

In this wizard step, (Figure 2), you define the period over which the calibration should be performed for each observation site.

The following dates are required:

• The start and end dates for the overall simulation, which is constrained by the available input data (typically climate records) - with the goal of providing a warm up period for each gauge. The Simulation End Date is automatically set to the latest Calibration End Date; and

• The Start and End dates for calibration at each site, which are constrained by the available observation data for the site - allows the calibration to take place over a subset of the observed data. This might be in order to save some data for validation, or to filter out poor quality observations.

Source provides Start date and End date values which represent the lowest common denominator of the time series you loaded in the preceding step and all other input time series that have been used within the Source model (such as rainfall and evaporation data). Source also proposes a warm-up period; calculation of the objective function for the calibration commences after the simulation reaches the Warmup end date. You can adjust all dates as required and then click Next.

As an example, consider the calibration of a model with a climate record from 1 January 1950 to 31 December 1999 (as shown in Figure 3). There are two observations sites, with the following observed streamflow records:

• S1: 1 January 1960 - 31 December 1989; and
• S2: 1 January 1975 - 31 December 2010.

Figure 3. Timeline for calibration period

Consider the following valid calibration periods:

• S1 starts on 1/1/1960 and ends on 31/12/1989; S2 starts on 1/1/1975 and ends on 31/12/1999 - for a simulation start date of 1/1/1959 and end date of 31/12/1999, the calibration periods of both S1 and S2 are valid as they fall within the simulation period; and
• S1 and S2 have the same start (1/1/1975) and end (31/12/1989) dates; the simulation start and end dates are 1/7/1974 and 31/12/1989 respectively - a subset of both S1 and S2 are used with a 6-month warm-up period for both records.

The period from the start date to the warm up end date should be sufficiently long to ensure that the soil moisture and groundwater stores represented within the rainfall runoff models have a sufficiently long warm up run period. That way, the volume of water in each of the stores will not be influenced by the volume that was set within those stores at the start of the warm up period. For most catchments and models, a warm up period of between 3 and 12 months should be sufficiently long for this to occur.

Anchor
Working with metaparameters Working with metaparameters

Working with metaparameters

In the calibration tool, metaparameters are parameters defined specifically for the purposes of calibration, with each metaparameter mapping to one or more parameters in the underlying model. In this way, metaparameters allow you to selectively reduce the number of independent model parameters that need to be calibrated. Each metaparameter defines the allowable ranges across which each of the associated model parameters can be varied as the calibration executes. Any model parameter which is not associated with a metaparameter will not be varied during a calibration run and will remain at the same value as it was set to during the set up of the model via the Edit menu before entry into the wizard.

Metaparameters can be extremely fine-grained with a one-to-one relationship between a metaparameter and a model parameter, coarse-grained where a single metaparameter controls all the like-named model parameters in the model, or anything in between. Metaparameters are created by grouping one or more like-named model parameters and, accordingly, it can be helpful to treat metaparameter and group as synonyms.

In Source, each node, link, catchment and functional unit is a model in its own right, with its own, independent parameters. Thus, even when a catchment model is built using a rainfall runoff model with a small set of parameters, such as GR4J which has four parameters, the application of the model across multiple functional units in many catchments has a multiplicative effect on the number of model parameters.

Defining metaparameters

The Metaparameter definition interface (Figure 4) is divided into a display table of metaparameters and three functional panel frames.

• Existing Meta Parameters - displays all exiting metaparameters in the table. Metaparameters can be created from the previous version of Source,  or just created.
• Meta Parameter Creation - This panel frame can be used to create the metaparameter for different model types:

Define from catchment model: Clicking on this button will display the interface (Figure 5) to define the metaparameter for the catchment rainfall runoff models or Constituent models. The routing link associated with a catchment model can be also defined here;

Define from Links: Clicking on this button will display the interface (Figure 6) to define the metaparameter for the link routings; and

Define Custom: The user can use this button to create the metaparameters undefined by Source.

• Selected Meta Parameter(s) - This panel can manage the existing metaparameter records in the table of Existing Meta Parameters:

Edit: this will open the interface (Figure 7) for editing the selected metaparameter record in the table in Existing Meta Parameter; and

Delete: this will delete all selected metaparameter records from the table of Existing Meta Parameter.

• Others – this panel provides the function to create/edit the other items involved in the calibration such as input sets calibration property, enforced relationship:

Edit Calibration Input Set properties: This button allows to open the interface of Calibration Input Set properties to edit or enter the Input Set calibration properties; and 

Edit Relationships: This button will define the enforced relationship between the metaparameters during the calibration. Such relationship normally has an evidence or physical meaning.

Some functions will be detailed in the separated sections late in this page.

Once you have finished defining and editing metaparameters, clicking on Next to go to next step (Figure 11).

Figure 4. Calibration Wizard (Metaparameter definition)

In the calibration tool, metaparameters are parameters that are defined specification for the purposes of calibration, with each metaparameter mapping to one or more parameters in the underlying model. In this way, metaparameters allow you to selectively reduce the number of independent model parameters that need to be calibrated. Each metaparameter defines the allowable range across which each of the associated model parameters can be varied as the calibration executes. Any model parameter which is not associated with a metaparameter will not be varied during a calibration run and will remain at the same value as it was set to during the set up of the model or via the Edit menu before entry into the wizard.

Creating metaparameters for Rainfall Runoff models or Constituent models

To create a metaparameter for a rainfall runoff model/Constituent model:

Filter:

• Clicking on the button Define from catchment model (Figure 4) to display the interface (Figure 5);
• Select one from Rainfall Runoff or Constituents  in the top left corner (Figure 5) for the calibrated model (rainfall runoff model or constituent model);
• Select one or more functional units (Figure 5). You can add functional units to the selection using either the shift or control keys. You can select all functional units by clicking AllFUs; and 
• Select one or more sub-catchments. (a) If you select a node in Figure 1 ( and only one gauge node in Figure 1), only associated upstream catchments are displayed on the map in Figure 5 and you can easily to define to catchment(s)  using Select All button. (b) if you want to define a point from the whole catchment on Figure 5, you can add sub-catchments to the selection using the Control key or by dragging an extent across the map of catchments. Whenever at least one FU and one sub-catchment is selected, a list of available model parameters will be displayed in the right panel ( i.e.. Available Parameters to choose from). A model parameter is only available in the right panel if it is not already associated with a metaparameter.

Group:

• From the list of available model parameters, select one or more like-named parameters. You can add model parameters to the selection using either the Shift or Control keys;
• Click Group Selected. This creates a metaparameter and associates it with the selected model parameters. The selected model parameters are then removed from the list of available model parameters; and
• In addition to the process outlined above, you can also create metaparameters using the AutoGroup All button. Auto-grouping creates one metaparameter for each distinct type of model parameter. After clicking AutoGroup All, the list of available model parameters will always be empty – for that combination of functional units and sub-catchments.

Result:

• Clicking on AutoGroup All or Group Selected will also close the interface ( Figure 5) and created metaparameter (s) will be added to the table of Existing Meta Parameter (Figure 4).

Figure 5. Calibration Wizard (Create metaparameter for catchment model)

You continue the process of creating metaparameters until all of the model parameters you wish to have varied during the calibration run have been associated with metaparameters, or the list of available model parameters is empty. Remember that the available parameters list depends on your FU and sub-catchment selections.

Creating metaparameters for the Link routing