Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: updated intro training link to self hosted page

The Results Manager allows you to examine and analyse scenario results in Source. Results can be evaluated graphically in charts, numerically in tables or as statistics, and they can also be filtered and manipulated using transforms. With custom charts, you can also view and statistically analyse multiple results within and between runs, compare results to external data sources and automatically update with the latest results each time a model is run.

Info
iconfalse

Note: Results Manager is best learned through doing - please see the introductory training.

Results Manager overview

Every scenario run generates a result set – a distinct collection of results recorded for that run, that can be viewed using the Results Manager. By default, on completion of a scenario run, the Results Manager automatically opens in a new window. You can choose not to open the window via Tools » Application settings and selecting disable Pop-up Results manager after run completion.

The Results Manager can also be accessed via:

  • View » Results Manager...; or
  • By pressing F10 on your keyboard. Pressing F10 again changes focus to the main Source window. 

The Results Manager window (Figure 1) is comprised of:

  • The tree menu, on the left, displaying results for all scenarios run in the current Source session, previously saved runs, and any loaded result files (*.res.csv) or time series (Data file formats);
  • The toolbar with buttons for data import and export, going to results, and configuring transformations;
  • The search bar, which allows you to search for results - start typing the search string for results to appear as you type;
  • One or more result tabs that contain view tabs for examining and analysing results in different ways (eg. chart, statistics); and
  • The customs charts and statistics pane, below the tree menu, lists any custom charts and special statistics you have generated.

These are all described in more detail below.

Info
iconfalse

Note: Custom chart refers to the relevant chart, table, transforms, data and statistics, not just the graph shown in the chart tab. 

Table of Contents
maxLevel4

Anchor
FigureResultsManagerMain
FigureResultsManagerMain
Figure 1. Results Manager


Info
iconfalse
Note: You can rename, delete and view data in Results Manager, but it cannot be edited.

Anchor
TableOfTerms
TableOfTerms
Results Manager common terms

In Results Manager there are a number of key terms. These are mostly related to naming conventions for results, groups of results, and how those results are manipulated. See Table 1 below.

Table 1. Commonly used terms
TermMeaning
Scenario runA run of a scenario in Source. A scenario run may also be referred to as a 'run' and the most recent is referred to at 'Latest run' in the tree menu in Results Manager.
ResultA single recorded output from a scenario run (eg. Total Inflow Volume, Figure 1) or a column in a loaded time series file.
ResultsMore than one result, eg. results in a custom chart can be viewed in chart or table views.
Result setA logical grouping of results. This is commonly used to discuss results from a scenario run, but could also be applied to, for example, downstream flow from multiple runs. If you load a result file, it contains a result set.
Result fileA *.res.csv file. This is a Source-generated text-based csv file that also contains metadata, which allows you to import the file back in to Source and maintain data structure and units.
Result tabDisplays result(s) in the central workspace of Results Manager, this is usually a single time series result or a custom chart, but can also be a unique view specific to a non-time series result or user-generated statistics (Figure 1). See Result Tabs.
View tabDisplays a view of the selected result or custom chart, such as Chart, Table or Statistics. See Result tabs.
Time seriesData that has been measured at a regular time interval (eg. observed daily flow). There are several time series file types compatible with Source (refer to a complete list in Data file formats). One of the most commonly used time series file types is *.csv.
Custom chartA custom chart is a user-created collection of two or more time series results that allows comparison between those results graphically in charts, numerically in tables or by comparative statistics. See Custom Charts.
TransformsTransforms allow you to modify (or transform) the view of the data. Once generated, they can be used as a template for another action, such as applying to custom charts. See Transforms.
Configuration

The configuration of Results Manager includes all custom charts, transforms, user-generated statistics and file locations of loaded time series and results files. You can save your current configuration as a Results Manager configuration file (*.dora), allowing you to load that configuration in the future. See Configurations.

Central workspace

The central workspace in the Results Manager window is where you will see your charts, tables, statistics and other results. For some displays, there are right click contextual menus within the central workspace. The display in the central workspace is specific to the item you have selected in the tree menu or custom chart and statistics pane - if no item has been selected, no chart will be visible.

Anchor
TreeMenu
TreeMenu
Tree menu

The tree menu on the left side of the Results Manager window (Figure 2) shows all result sets from the current session, any saved results from previous runs, along with any loaded result files (*.res.csv) and time series files (Data file formats).

Anchor
NamingConventions
NamingConventions
Once run, results from a scenario appear grouped together under Latest run and as a separate item grouped by scenario in run order. Those previous runs are named using the following convention, where # represents the run number from the current session:

  • (#) Scenario name – eg. (2) Stage A, means that it is the second run of the current session, and that the scenario run was called Stage A. 
  • (#) Scenario Input Set name – when there is only one scenario, and more than one Scenario Input set.
  • (#) Scenario name (Scenario Input Set name) – when there is more than one scenario and the chosen scenario has more than one scenario input set.

The Results Manager tree menu is shown in Figure 2.

Info
iconfalse
Note: Once a scenario run is complete, clicking on a node in the schematic or geographic editor will highlight that node's results in the Results Manager tree menu for the currently active scenario result set. If no scenario result set is active (eg. because a custom chart is selected), the Latest Run result set is used.


Anchor
ResultsManagerTreeMenu
ResultsManagerTreeMenu
Figure 2. Results Manager, tree menu

The following should also be noted. 

  • The icon displayed after a scenario run corresponds to the notification level in the Log Reporter. The icon indicates that the run contains warnings and theicon indicates that the run was incomplete because of a fatal error.
  • Previously saved result sets and time series can be loaded (see Loading Results), and appear under Loaded Result Files and Loaded Time Series, respectively.
  • You can rename loaded result sets, time series and runs (except the latest run) that appear in the tree menu. However, this does not change the name of results grouped under the result set, time series or run - they will retain the original name, and any view tab will list the full name of the result as it was loaded in the Source project. 
  • Anchor
    SaveRunInProject
    SaveRunInProject
    Run results can be saved with the project using the Save Run in Project contextual menu item (in upper left contextual menu, Figure 2). This saves the run results and loads them automatically when the project is next opened. Note that this will increase the project's file size, particularly if you record many results or the model has a long run time. Runs that will be saved or have previously been saved in the project are indicated by a save icon next to the scenario name (Figure 3). If you have previously chosen to save a run in the project and you no longer wish to do so, you need to delete the run (right click on the run and choose Delete from the contextual menu). 
Figure 3. Tree Menu showing a run saved in the project

Anchor
ResultTabs
ResultTabs
Result tabs

The central workspace of Results Manager contains result tabs, which can display:

  • A single time series result (as in Figure 1);
  • A custom chart, which allows the comparison of multiple time series; 
  • A unique view specific to a non-time series result (eg. Scenario Mass Balance, Travel Time, Borrow and Payback); and
  • A unique view of statistics generated for either an entire scenario (eg. Mass Balance) or for a single result from different scenarios (eg. Annual Descriptive Statistics).

Apart from results with unique views, each result tab contains the following view tabs:

  • The Chart tab provides a graphical representation of data, which can be customised using the Chart tab toolbar (refer to Charts);
  • The Table tab displays the results in a table and allows you to query the table using advanced search functionality (refer to Table); 
  • The Transforms tab provides a list of all transforms that have been applied to the result(s) in that result tab (refer to Transforms);
  • For custom charts only, the Data tab displays what results the custom chart contains in a tree menu; and
  • The Statistics tab shows a statistical analysis of the data – Univariate statistics are displayed for a single result, Bivariate andUnivariate comparison statistics are displayed for custom charts (refer to Statistics).

To use result tabs:

  • Shift-click on a result (in the tree menu), or a custom chart or statistic (in the pane below the tree menu) to open that item in a new result tab.
  • Click and drag a result tab to move it to a new result window, which will include the central workspace and view tabs depending on the type of view (see unique views above). You can repeat this to open more result windows.
  • Once you've done this, and have one (or more) result window open along with the Result Manager window (which has a tree menu), you can Shift+Click on a result in the chart tab of any of these windows and drag it to another window to copy the result to the other window. This is synonymous to clicking the Drag button on the Chart toolbar and then dragging the result to another window. The result window or tab to which you dragged the result will automatically change to a custom chart, if it is not one already. 

Custom charts and statistics pane

The custom charts and statistics pane is located to the left of the central workspace, below the tree menu panel. It lists the custom charts and user-generated statistics that have been created in the Source project. The pane and its contextual menus are shown in Figure 4. The icon beside an item in the pane indicates that a transform has been applied. Note that Add Transform is only shown if there are available transforms.

Figure 4. Results Manager, Custom chart and statistics pane with defined transforms available

Anchor
ResultsManagerToolbar
ResultsManagerToolbar
Results Manager toolbar

The Results Manager toolbar (Figure 5) is located near the top of the Results Manager window and includes load, save, search and transform options.

You can load:

  • a folder of result files (Ctrl+R);
  • a time series (Ctrl+O); or
  • a configuration file (Ctrl+L).

You can also save the current configuration (Ctrl+S) and set up your data transforms (Ctrl+Shift+T). To go to a result in the tree menu, select the search icon and then start typing a search string for results from the tree menu for the currently active scenario to appear as you type, see Searching for results

Figure 5. Results Manager toolbar

Anchor
CustomCharts
CustomCharts
Custom charts

Custom charts allow you to examine and statistically analyse multiple results within and between runs, compare results to external data sources and automatically update with the latest results each time a model is run. Whenever you want to compare two or more results, you need to create a custom chart.

Creating custom charts

A custom chart comparing Results A and B can be created using one of two methods:

  1. Right click on Result A and choose Add to Custom chart » New Custom chart from the contextual menu. This will create a new custom chart in the panel below (named Custom Chart #2 for example); and
  2. Right click Result B and choose Add to Custom chart » Custom chart #2.

OR

  1. Click on the result in the tree menu to display Result A in the chart tab; and
  2. Then, Ctrl-click on Result B.

Additional results can be added to a custom chart by repeating the second step of either of these methods. There is no limit to the number of results that can be added to a custom chart, but only time series data can be added to custom charts. Note that if a custom chart is active when you Ctrl-click on a result, the result will be added to that custom chart. Otherwise, a new custom chart will be created. Ctrl-clicking on a result that is already in a custom chart will delete it.

Info
iconfalse

Note: if you add a result from the Latest Run result set to a custom chart, the custom chart will update that result whenever you next run a scenario, including in a subsequent Source session.

Custom chart features

The view tabs in a custom chart differ from those of a single result as summarised below.

  • Chart tab - each result is displayed in the chart and listed in the legend next to its colour, along with a checkbox, which can be toggled off to hide the result. When a custom chart is created, additional chart types become available for comparing results to each other, e.g. difference, double mass, and scatter plots. Refer to Charts and Chart Types.
  • Table tab - each result is displayed in its own column relative to a date column that covers the date range for all of the data combined.
  • Transforms tab - if you apply a transform to a custom chart it is applied to all results in the custom chart. Refer to data manipulation.
  • Data tab - this tab only exists in a custom chart. It displays the results that have been added to the custom chart in the tree menu in the panel on the left of the Results Manager window.
  • Statistics tab - in addition to univariate statistics, bivariate statistics are displayed. By default the statistics are calculated for the overlapping date range between all results in the custom chart. Refer to Statistics.

Custom charts are automatically saved when you save your project. This will save which results are in your custom chart along with the transforms you have applied, the chart type, and any other options you have configured (eg. short header in Tables). However, results themselves are not saved unless you choose to save the run(s) that generated the results. When you next open your project, the custom chart will be empty until a result is generated that exactly matches the name of a result used in the custom chart, ie. the scenario name and run number have to match, so you need to run the scenarios in the same order. If results from Latest Run are used to generate a custom chart, it will be populated as soon as you run any scenario. Therefore, if you want to return to an analysis in a custom chart after closing a project, it is best to use either results from Latest Run or to save the run in the project.

Info
iconfalse
Note: Results Manager settings (including custom charts) are saved with the Source project.

Result management

Info
iconfalse
Note: For simplicity, any loaded files (*.res.csv or time series) are called results.

Anchor
LoadingResults
LoadingResults
Loading results

Results can be loaded into the Results Manager using:

  • The File menu;
  • The Results Manager toolbar; or
  • The equivalent Hot Keys (Figure 5). 

There are two ways to load data:

  • Load Results Files – loads all *.res.csv files in a specified folder (even if there are other time series in the folder). This is non-recursive, ie. it does not load files within sub-folders. The loaded files are listed under Loaded Result Files at the bottom of the tree menu.

OR

  • Load Time Series – loads any time series file type compatible with Source (refer to a complete list in Data file formats). Multiple files can be selected and imported at once. The imported files are listed under Loaded Time Series at the bottom of the tree menu. Just like data sources, if a file has multiple columns, all columns will be imported. Unlike data sources, there is no choice to only import some of the data columns.

To set units on loaded time series, select the time series file name. The right side of the Results Manager window will display the Unit Preferences options as shown in Figure 6 below. To select the preferred unit, click on 'no unit selected' and then choose from the list of available units.

Figure 6. Unit Preferences for Loaded Time Series



Anchor
ExportingResults
ExportingResults
Exporting results

To export a result, result set or custom chart, right click on it in the tree menu panel and choose one of the two export options from the contextual menu:

  • Export Time Series – this allows you to save the result(s) as a *.res.csv file, which can then be re-imported for future use maintaining the same data structure; or
  • Export to Excel – opens the result(s) in Excel, allowing you to save a *.csv file.

For single results, both export methods allow you to export for either the Current Run only or All Runs. All Runs will export that result for all runs in the current session. 

Info
iconfalse
Note: When you right click on any entry in the tree menu, apart from a single result, you can choose to export data for all series beneath it. So to save all results for a scenario, right click on the scenario name. This will export results for a scenario, which is not the same as saving a run in your project.

Anchor
SearchingForResults
SearchingForResults
Searching for results

You can search for specific results without navigating through the tree. Searches can be undertaken in one of two ways:

  • Enter the result name in the Search field above the tree menu to filter the tree; or
  • Click on the Go to a result in the tree (magnifying glass) icon in the toolbar (synonymous to entering Ctrl+T).

Each of these methods searches the results differently:

  • The Search field searches all levels of the tree menu (including loaded results or time series) for the entered phrase and hides any result that does not contain that phrase anywhere in its path. For example searching for 'Inflow' will leave unhidden all results with the word Inflow in their path, which includes all results with inflow in their name (eg. Total Inflow Volume), and also all results for all Inflow nodes. Therefore, this search field is most useful when searching for a specific result (eg. Downstream Flow Volume) or node name (eg. Fish Creek Confluence).
  • The Go To option allows you to search the currently active scenario using the wild card * to represent any character. If there is no active scenario (eg. because you have selected a custom chart), it will search in the Latest Run. For example, "Conf*Inf" can be used to find the Total Inflow Volume for all Confluence nodes, regardless of their node names. You can then select one of the returned results and it will be displayed. This dialog displays a maximum of 16 results, even if there are more results that meet the search criteria.

Anchor
Configurations
Configurations
Configurations

A Results Manager configuration file (*.dora) contains a record of:

  • All custom charts;
  • Transforms and where they are used;
  • User-generated statistics;
  • File locations of loaded time series and results files;
  • Water Year start date (set in Project Options);
  • Short table headers (set on Table view tab); and
  • Visible decimal places (set in Application settings).

You can: 

  • Save the current configuration of the Results Manager by selecting Save Configuration from the File menu or toolbar; and
  • Load a previously saved configuration using the Load Configuration from the File menu or toolbar. The integrity of a configuration file depends on the run order being consistent.
Info

Note: Saving your Source project file will save the same information, but will also save any run results that you have selected using Save Run In Project

Hot Keys

Choose Help » HotKeys to view a list of shortcuts that can be used within the Results Manager (Figure 7).

Figure 7. Results Manager, hotkeys

 

Stand-alone mode

Results Manager can be run independently of Source, allowing you to analyse previously saved Source results or time series from other programs without opening Source. You can launch Results Manger in stand-alone mode from your Windows Start menu, the standard location is:

Start » All Apps » eWater Source n.n.n.n » Results Manager n.n.n.n

The only difference between stand-alone mode and running Results Manager within Source is a Configuration Options button in the Results Manager toolbar (Figure 8). This opens a Configuration dialog (Figure 8) that allows you to configure the following options:

  • Enable Rounding – Determines whether the values in the Table and Statistics views are rounded to the number of places entered in Visible Decimal Places.
  • Short Table Headers – Toggles on or off the display of long or short names of results in the Table view. Can also be enabled on the Table view tab.
  • Water Year Start – Allows you to set the default water year start date, which affects the calculation of statistics in the Statistics view.
Figure 8. Results Manager toolbar, stand-alone mode