Results Manager

The Results Manager allows you to examine and analyse scenario results in MUSIC X. Results can be evaluated graphically in charts, numerically in tables or as statistics, and they can also be filtered and manipulated using transforms. With custom charts, you can also view and statistically analyse multiple results within and between runs, compare results to external data sources and automatically update with the latest results each time a model is run.

Results Manager overview

Every scenario run generates a result set – a distinct collection of results recorded for that run, that can be viewed using the Results Manager. By default, on completion of a scenario run, the Results Manager automatically opens in a new window. You can choose not to open the window via Tools » Application settings and selecting disable Pop-up Results manager after run completion.

The Results Manager can also be accessed via:

• View » Results Manager...; or
• By pressing F10 on your keyboard. Pressing F10 again changes focus to the main MUSIC X window. 

The Results Manager window (Figure above) is comprised of:

• The tree menu, on the left, displaying time series results for all simulations run in the current Source session, previously saved simulation results, and other time series data imported from file;
• The Toolbar with buttons for data import and export, going to results, and configuring transformations;
• The search bar, which allows you to search for results - start typing the search string for results to appear as you type;
• One or more result tabs that contain view tabs for examining and analysing results in different ways (eg. chart, statistics); and
• The customs charts and statistics pane, below the tree menu, lists any custom charts and special statistics you have generated.

These are all described in more detail below.

Note: Custom chart refers to the relevant chart, table, transforms, data and statistics, not just the graph shown in the chart tab. You can rename, delete and view data in Results Manager, but it cannot be edited.

Results Manager common terms

In Results Manager there are a number of key terms. These are mostly related to naming conventions for results, groups of results, and how those results are manipulated. See Table below.

Scenario Run Metadata

Each scenario run is issued a unique ID. When export results from a scenario run, the exported .res.csv files also includes this ID. The metadata associated with this run ID is available in results manager (See Figure 2 below). The metadata can be saved by supported file types. 

Toolbar

The Results Manager toolbar (Figure below) is located near the top of the Results Manager window and includes load, save, search and transform options.

You can load a:

• folder of result files (Ctrl+R);
• time series (Ctrl+O);
• Source Database (Ctrl+D); or
• configuration file (Ctrl+L).

You can also save the current configuration (Ctrl+S) and set up your data transforms (Ctrl+Shift+T). To go to a result in the tree menu, select the search icon and then start typing a search string for results from the tree menu for the currently active scenario to appear as you type, see Searching for results. 

Central workspace

The central workspace in the Results Manager window is where you will see your charts, tables, statistics and other results. For some displays, there are right click contextual menus within the central workspace. The display in the central workspace is specific to the item you have selected in the tree menu or custom chart and statistics pane - if no item has been selected, no chart will be visible.

Tree menu

The tree menu on the left side of the Results Manager window (Figure 4) shows all result sets from the current session, any saved results from previous runs, along with any loaded result files (*.res.csv) and time series files.

Once run, results from a scenario appear grouped together under Latest run and as a separate item grouped by scenario in run order. Those previous runs are named using the following convention, where # represents the run number from the current session:

• (#) Scenario name – eg. (2) Stage A, means that it is the second run of the current session, and that the scenario run was called Stage A. 
• (#) Scenario Input Set name – when there is only one scenario, and more than one Scenario Input set.
• (#) Scenario name (Scenario Input Set name) – when there is more than one scenario and the chosen scenario has more than one scenario input set.

The Results Manager tree menu is shown in Figure 2.
Note: Once a scenario run is complete, clicking on a node in the schematic or geographic editor will highlight that node's results in the Results Manager tree menu for the currently active scenario result set. If no scenario result set is active (eg. because a custom chart is selected), the Latest Run result set is used.

Figure 4. Results Manager, tree menu

The following should also be noted. 

• The icon displayed after a scenario run corresponds to the notification level in the Log Reporter. The icon indicates that the run contains warnings and the  icon indicates that the run was incomplete because of a fatal error.
  
• Previously saved result sets and time series can be loaded (see Loading Results), and appear under Loaded Result Files and Loaded Time Series, respectively.
• You can rename loaded result sets, time series and runs (except the latest run) that appear in the tree menu. However, this does not change the name of results grouped under the result set, time series or run - they will retain the original name, and any view tab will list the full name of the result as it was loaded in the Source project. 
• Run results can be saved with the project using the Save Run in Project contextual menu item (in upper left contextual menu, Figure 2). This saves the run results and loads them automatically when the project is next opened. Note that this will increase the project's file size, particularly if you record many results or the model has a long run time. Runs that will be saved or have previously been saved in the project are indicated by a save icon next to the scenario name (Figure 5). If you have previously chosen to save a run in the project and you no longer wish to do so, you need to delete the run (right click on the run and choose Delete from the contextual menu). 

Figure 5. Tree Menu showing a run saved in the project

Result tabs

The central workspace of Results Manager contains result tabs, which can display:

• A single time series chart (as in Figure 1);
• A custom chart, which allows the comparison of multiple time series; 
• A unique view specific to a non-time series result (eg. Scenario Mass Balance, Travel Time, Borrow and Payback); and
• A unique view of statistics generated for either an entire scenario (eg. Mass Balance) or for a single result from different scenarios (eg. Annual Descriptive Statistics).

To use result tabs:

• Shift-click on a result (in the tree menu), or a custom chart or statistic (in the pane below the tree menu) to open that item in a new result tab.
• Click and drag a result tab to move it to a new result window, which will include the central workspace and view tabs depending on the type of view (see unique views above). You can repeat this to open more result windows.
• Once you've done this, and have one (or more) result window open along with the Result Manager window (which has a tree menu), you can Shift+Click on a result in the chart tab of any of these windows and drag it to another window to copy the result to the other window. This is synonymous to clicking the Drag button on the Chart toolbar and then dragging the result to another window. The result window or tab to which you dragged the result will automatically change to a custom chart, if it is not one already. 

Custom charts and statistics pane

The custom charts and statistics pane is located to the left of the central workspace, below the tree menu panel. It lists the custom charts and user-generated statistics that have been created in the Source project. The pane and its contextual menus are shown in Figure 6. The icon beside an item in the pane indicates that a transform has been applied. Note that Add Transform is only shown if there are available transforms.

Figure 6. Results Manager, Custom chart and statistics pane with defined transforms available

Managing Results

Note: For simplicity, any loaded time series (*.res.csv or other formats) are called results.

Data Import and Export

Time series data can be imported from, and exported to, a variety of file formats, as discussed in Data Import and Export.

Searching for results

You can search for specific results without navigating through the tree. Searches can be undertaken in one of two ways:

• Enter the result name in the Search field above the tree menu to filter the tree; or
• Click on the Go to a result in the tree (magnifying glass) icon in the toolbar (synonymous to entering Ctrl+T).

Each of these methods searches the results differently:

• The Search field searches all levels of the tree menu (including loaded results or time series) for the entered phrase and hides any result that does not contain that phrase anywhere in its path. For example searching for 'Inflow' will leave unhidden all results with the word Inflow in their path, which includes all results with inflow in their name (eg. Total Inflow Volume), and also all results for all Inflow nodes. Therefore, this search field is most useful when searching for a specific result (eg. Downstream Flow Volume) or node name (eg. Fish Creek Confluence).
• The Go To option allows you to search the currently active scenario using the wild card * to represent any character. If there is no active scenario (eg. because you have selected a custom chart), it will search in the Latest Run. For example, "Conf*Inf" can be used to find the Total Inflow Volume for all Confluence nodes, regardless of their node names. You can then select one of the returned results and it will be displayed. This dialog displays a maximum of 16 results, even if there are more results that meet the search criteria.

Results Manager Configurations

A Results Manager configuration file (*.dora) contains a record of:

• All custom charts;
• Transforms and where they are used;
• User-generated statistics;
• File locations of loaded time series and results files;
• Water Year start date (set in Project Options);
• Short table headers (set on Table view tab); and
• Visible decimal places (set in Application settings).

You can: 

• Save the current configuration of the Results Manager by selecting Save Configuration from the File menu or toolbar; and
• Load a previously saved configuration using the Load Configuration from the File menu or toolbar. The integrity of a configuration file depends on the run order being consistent.

Note: Saving your Source project file will save the same information, but will also save any run results that you have selected using Save Run In Project. 

Hot Keys

Choose Help » HotKeys to view a list of shortcuts that can be used within the Results Manager (Figure 7).

Figure 7. Results Manager, hotkeys

Stand-alone mode

Results Manager can be run independently of Source, allowing you to analyse previously saved Source results or time series from other programs without opening Source. You can launch Results Manger in stand-alone mode from your Windows Start menu, the standard location is:

Start » All Apps » eWater Source n.n.n.n » Results Manager n.n.n.n

The only difference between stand-alone mode and running Results Manager within Source is a Configuration Options button in the Results Manager toolbar (Figure 8). This opens a Configuration dialog (Figure 8) that allows you to configure the following options:

• Enable Rounding – Determines whether the values in the Table and Statistics views are rounded to the number of places entered in Visible Decimal Places.
• Short Table Headers – Toggles on or off the display of long or short names of results in the Table view. Can also be enabled on the Table view tab.
• Water Year Start – Allows you to set the default water year start date, which affects the calculation of statistics in the Statistics view.

Figure 8. Results Manager toolbar, stand-alone mode