Statistics

Using the Statistics tab of the Results Manager, you can carry out several types of statistical analyses on scenario results. For simplicity, they can be categorised into two groups:

• Auto-generated: This displays basic statistical data for a single result (univariate statistics) or for multiple results via custom charts (bivariate statistics); and
• User-generated: You can generate statistics for either an entire scenario (eg. Mass Balance) or for a single result from different scenarios (eg. Annual Descriptive Statistics).

For all types of analyses, you can export results (to *.csv) or view them in Excel.

Auto-generated statistics

If a single result is selected from the left-hand tree menu of the Results Manager window, the statistics tab will show univariate statistics (Figure #1). For multiple results in a custom chart, the statistics tab shows both univariate comparison and bivariate statistics for two of the results in the chart.

For statistics for a single result, you can:

• Choose the period type (Table 1) used to calculate the statistics from the drop down menu;
• Change the Water Year Start from the default date (the default is set in Project Options);
• View the result used to calculate the statistics under Data;
• View the statistics themselves.

For statistics of multiple results, you can also:

• Toggle on or off two Statistics Options for how the two results are compared (Table 3);
• View and change the result used as the X Data (reference) and the Y Data for statistic calculations (in Figure #2, this is Downstream Flow Volume); and
• Select to view either Univariate Comparison or Bivariate statistics for the two results.

Table 1. Basic statistical analysis, period types

Table 2. Multiple results, parameters

Figure 2. Univariate statistics (multiple results), yearly summary

Naming conventions of difference, double mass and scatter statistics

If you have a custom chart with a chart type of difference, double mass or scatter, then the following naming conventions are used in the statistics tab (Figure 3):

• Each result in the custom chart is given a letter. This is shown in a Legend in the bottom left of the statistics tab.
• The series used as a reference in the chart is always given letter A. To change the reference series, go to the Chart view then select Chart Settings » Charts and modify Chart Type Reference Series.
• Under X Data (Reference) and Y (Data), the results are listed using the letters to represent each result.

For example, in Figure 3, A - B corresponds to the difference between Crab Creek's Downstream flow volume (result A) and Fish Creek's Downstream Flow Volume (result B). 

Figure 3. Statistics, Difference Chart Type

Univariate statistics

You can view univariate statistics for a single result (as shown in Figure #1) or for each individual result when multiple results are combined (Figure #2). A brief description of each univariate statistic is given in Table 1.

Table 1. Univariate Statistics

Bivariate statistics

When two or more results are in a custom chart, a set of bivariate statistics is automatically generated and can be viewed on the Bivariate Statistics tab (Figure #4).  A brief description of each bivariate statistic is given in Table 4, where:

• NSE (Nash-Sutcliffe Efficiency) measures the relative magnitude of the model error variance compared to the measured data variance. It can be applied at any time step size (eg. daily, monthly). See Nash-Sutcliffe Efficiency
• The NSE of Flow Duration measures the relative magnitude of the model error variance compared to the measured data variance. It can be applied at any time step size (eg. daily, monthly). See Flow Duration.
• Relative Bias measures the magnitude of the model errors relative to the magnitude of the observations. See Relative Bias.
• Bias Penalty is log transformation of the absolute value of the relative bias. In Source, the bias penalty is always used in combination with the NSE and is not available on its own. It is designed to be used in model calibration to penalise biased solutions. See Bias Penalty.

For more detailed information on bivariate statistics, see Bivariate Statistics - SRG.

Parameter selection for bivariate statistics is the same as for a univariate comparison. If you select any period type other than Total, or you have disabled Calculate using overlapping data only and your start and/or end dates do not match, a subset of the statistics are available:

• Pearson's Correlation (r);
• NSE Daily;
• Volume Bias (%);
• Values used; and
• Values not used.

Table 4. Bivariate Statistics

Figure #4. Bivariate statistics, Total period

User-generated statistics

User-generated statistics include Annual Descriptive Statistics and Mass Balance. Compared to the auto-generated statistical analyses, these provide a broader range of statistical data. 

Annual descriptive statistics

Figure #5 shows annual descriptive statistics for a single result:

• The upper left pane shows the overall statistical data;
• The  upper right pane allows you to select the relevant run/s; and
• The table at the bottom shows the resulting statistical data on an annual basis.

Figure #5. Annual descriptive statistics

Mass Balance

Results Manager provides a Mass Balance tool to check all of the mass flows entering, remaining in, or leaving the system. It is configurable for time period and for every part of the model that you can choose to enclose in a boundary. 

All individual inputs and outputs for the mass balance are specified by the user from the results of the model run, so before starting a mass balance check, ensure that all required results are recorded. Typical recorded results used in a mass balance check could be:

• All inflow volumes of inflow nodes, 
• storage volume, 
• storage rainfall volume, 
• storage evaporation volume, 
• storage infiltration volume, 
• extracted water from supply points, 
• downstream flow at the lowest point

Each inflow, storage, and outflow need to be added to the appropriate tab (Storage Start, Inflow, Outflows, Storage End).

To calculate mass balance, right click on a result set and select Statistics » New Mass Balance from the contextual menu (Figure #6). A new mass balance statistic will be created and listed under Statistics in the Custom Chart and Statistics pane, and the mass balance tool will be shown in the central workspace (Figure #6), where:

• The lower right pane allows you to select the result set(s) and time period used to calculate the mass balance;

• The upper right pane includes four tabs: Storage Start, Inflow, Outflows, Storage End; which will accept results listed in the upper left pane;

  Note: The Is Grouped checkbox relates to the mass balance report exported from the Results Manager using Export To Excel. If this box is unchecked, the result will be reported as its own separate total, otherwise a single grouped total will be given for each tab.
• The upper left pane lists the results from the selected result sets(s); right click on a result to add it to one of the four tabs in the upper right panel; and
• The lower left pane is the mass balance check, which will calculate the flow from the four tabs in the upper right panel.
  
  

Figure #6. Mass balance