The calibration wizard for catchments is a tool that provides computational assistance for calibrating catchments scenarios. It is not applicable to river management or river operations scenarios.

In summary, the steps involved in using the calibration wizard are:

• Launch Source, open your project, and then open the scenario you wish to calibrate.
• Start the wizard and work through its steps (see Calibration Wizard).
• Click Begin Analysis (Run) on the Simulation toolbar to perform a calibration run.
• Examine the results.

Calibration Wizard

To start the wizard, choose RR Calibration Analysis from the Select Analysis Type popup menu on the Simulation toolbar and then click Configure, which is also on the Simulation toolbar The wizard begins executing and leads you through a series of steps.

Select elements to record

In the first wizard step, you define the objective function to be used and associate observed data with one or more nodes in your model.

You use the Measurement functions popup (Figure 146) to define the objective function. Your choices are:

• Nash-Sutcliffe coefficient of efficiency (R2), daily and monthly variants;
• Relative Bias (total modelled error ÷ observed total streamflow);
• NSE-Bias penalty (Nash-Sutcliffe coefficient of efficiency with penalised bias solutions), daily and monthly variants; and
• Combined Nash-Sutcliffe and Flow duration curve (daily) variants. These are effectively combined objective functions.

Minimise Absolute Bias Minimise Absolute Bias between Observed and Modelled

Where you have multiple gauges,Calibration Weighting weights the objective function between those gauges. The default is 1 for each gauge, so each gauge would contribute equally to the overall objective function. You might use this in a situation where for instance, there are 20 years of data at one gauge and 10 years of data at another, then apply a weighting of 2 to the first gauge and 1 to the second gauge.

For each node for which you have observed data:

• Click the node in the geographic editor panel so that it becomes selected;
• Click Load observed data and choose a time-series file containing the observed data;
• Define the units of the time series using the Observed data units popup menu; and
• Click Apply.

To disconnect a node from a time series, click the X button to the left of the Load observed data button. To disconnect all time-series files from all nodes, click Clear.

When you have finished loading time series, click Next.

Define calibration period

In this wizard step (Figure 147), you define the period over which the calibration should be performed. Source provides Start date and End date values which represent the lowest common denominator of the time series you loaded in the preceding step. Source also proposes a warm-up period; calibration commences after the simulation reaches the Warmup end date. You can adjust all dates as required and then click Next.

Define metaparameters

Conceptually, metaparameters are containers that are associated with one or more model parameters. Each metaparameter defines the allowable range across which each of the associated model parameters can be varied as the calibration executes. Any model parameter which is not associated with a metaparameter will not be varied during a calibration run.

Metaparameters can be extremely fine-grained with a one-to-one relationship between a metaparameter and a model parameter, coarse-grained where a single metaparameter controls all the like-named model parameters in the model, or anything in between. Metaparameters are created by grouping one or more like-named model parameters and, accordingly, it can be helpful to treat metaparameter and group as synonyms.

To create a metaparameter:

• Select one or more functional units (Figure 148). You can add functional units to the selection using either the shift or control keys. You can select all functional units by clicking All FUs.
• Select one or more sub-catchments. You can add sub-catchments to the selection using the control key. Whenever at least one FU and one sub-catchment is selected, a list of available model parameters in the middle panel. A model parameter is only available if it is not already associated with a metaparameter.
• From the list of available model parameters, select one or more like-named parameters. You can add model parameters to the selection using either the shift or control keys.
• Click Group. This creates a metaparameter and associates it with the selected model parameters. The selected model parameters are then removed from the list of available model parameters.

Whenever you select more than one available model parameter with the intention of creating a group, you should ensure that the selected parameters either respond to the same range or that a sub-range exists which is common to the selected parameters. For example, if parameter A can be varied across the range 0.0 - 0.9 while parameter B can be varied across the range 1.0 - 10.0, and you group those into a single metaparameter, the results will be undefined. However, if A can be varied across the range 0.0 - 0.7 and B can be varied across the range 0.3 - 1.0, you can safely group A and B providing that you also edit the metaparameter to restrict its variability to the common subset range 0.3 - 0.7.

Warning Source does not calculate a common subset range for you. You must do it.

You adjust the range over which the associated model parameters are varied using the EditMetaParameter dialog (Figure 149). To adjust a range:

• Select the metaparameter to be edited in the list of metaparameters;
• Either double-click the metaparameter or click Edit;
• Adjust either or both of the Min and Max fields; then
• Click OK.

You can also create metaparameters using the Auto Group button. Auto-grouping creates one metaparameter for each distinct type of model parameter. After clicking Auto Group, the list of available model parameters will always be empty - for that combination of functional units and sub-catchments.

The name of each metaparameter is constructed from the name of the model parameters with which it is associated. This may produce naming collisions where the same model parameter name is used in different types of functional unit. To avoid a naming collision, you may need to rename one or more metaparameters to make them unique. To rename a metaparameter:

• Select the metaparameter to be renamed in the list of metaparameters;
• Either double-click the metaparameter or click Edit to open the EditMetaParameter dialog (Figure 149);
• Edit the Name: field so that the name is unique. Preserve the leading "$" because it is required.
• Click OK.

You continue the process of creating metaparameters until all of the model parameters you wish to have varied during the calibration run have been associated with metaparameters, or the list of available model parameters is empty. Remember that the available parameters list depends on your FU and sub-catchment selections.

Once you have finished defining and editing metaparameters, click Next.

Configure optimisation function

In this wizard step you choose an optimisation function (Figure 150) and then adjust its parameters. The available optimisation functions are:

• Shuffled Complex Evolution;
• Uniform Random Sampling; and
• Rosenbrock.

All optimisation functions allow you to set a limit on the number of iterations. Shuffled Complex Evolution offers additional parameters but these are experimental and should be left at their default values.

Click Finish to complete the wizard. If you need to make changes in any of the preceding steps, start the wizard again as described under Calibration Wizard, click Next until you reach the appropriate screen, edit as needed, then click Next until you reach the end of the wizard, and click Finish.

Calibration

Once a scenario has been configured for a calibration run using the wizard, you can start the calibration simulation run. Click Begin Analysis (Run) on the Simulation toolbar. The Simulation: Runner dialog (Figure 151) opens and the calibration run begins.

A calibration run is a series of iterations. By default, the iterations will continue until you click Stop. You can also impose a limit on the number of iterations in the Configure optimisation function step of the wizard.

The Simulation: Runner dialog shows two traces:

• The measure returned by the objective function from each iteration, with the leading edge of the trace being the most recent iteration; and
• The best measure obtained thus far. The value of the best measure is also shown in the legend.

Measure are expressed in the units returned by the objective function. For example, the Nash-Sutcliffe Coefficient of Efficiency (R2) can range from -∞ to +1.0. The term "best" also needs to be interpreted by reference to the objective function.

You can use the command line to do calibration and uncertainty in a catchments model. Refer to PEST calibration using the command line for more details.

Inspecting calibration results

When the calibration run completes, Source automatically selects the metaparameter values associated with the "best" iteration, applies those to the model, and executes a normal run using the same date-range as was defined at Define calibration period. This includes the warm-up period.

The results of that run appear on the Simulation: Runner dialog (Figure 152). Each result is an overlay of as-modelled vs as-observed. Using this dialog, you can also inspect the metaparameter values for each iteration. The information in this window is organised in rows and columns. Each row represents one iteration. The first column contains the measure returned by the objective function for each iteration. The remaining columns contain the metaparameter values applied to the model parameters for that iteration.

Once a simulation run has been completed, the rows of the results table are sorted by iteration order. You can click column titles to change the sort order. For example, clicking the left-most column title once sorts the table into ascending order of objective function measure. That is, the first row of the table will reflect the "worst" iteration. Clicking the same column title a second time will sort the table into descending order of objective function measure, so the first row will then contain the "best" iteration.

You can also export the results to Microsoft Excel. Additionally, you can select one row at a time and press Control+C to copy to the clipboard. Note that you cannot select individual cells.

Note Only single rows are copied to the clipboard. No column titles (metaparameter names) are copied.

You can select metaparameters and values and copy to the clipboard (Figure 153).