This section describes the Source processing tools that are installed by default, accessible via the Tools menu and sub-menus.
| Anchor | ||||
|---|---|---|---|---|
|
The Data calculator (accessed via Tools » Data Calculator Tool) can be used to analyse spatial raster data and time series through the use of simple arithmetic operators, or to compare data sets.
| Info | ||
|---|---|---|
| ||
| Note: Use can also drag and drop data to and from the Charting tool. First, choose Draggable from the contextual menu (as shown in Figure 1). Then, drag and drop the raster data or time series to the Charting tool. |
You can use rasters, time series or numbers as operands as follows:
- Drag and drop the raster data or time series into either of the two view controls (raster or time series are displayed in a small window called a View Control (Figure 1), or use a combination of a raster/time series and a number.
- Use the radio buttons to select either the view control or numeric values as left and right operands;
- Click one of the basic operations buttons (addition, subtraction, square root etc). The data calculator displays the operand you selected; and
- Click =, and the result, either raster/time series or numeric, appears in the results area on the right.
| Info | ||
|---|---|---|
| ||
| Note: Time series and raster data need to be in a format compatible with Source. Refer to Data file formats. |
The memory feature of the data calculator allows you to save previous results, either numeric or raster/time-series.
Click Memory to open the memory area. To save a result raster/time series into the memory, click on the 1st or 2nd operand, or result, view control, then drag and drop the contents into your desired memory view control. The label above each view control resides shows the mathematical operation leading to the result stored there. Figure 3 shows the memory area with several stored results.
| Info | ||
|---|---|---|
| ||
| Note: When you close the data calculator, the stored results are not saved. If you want to keep any of the rasters, right-click the raster, and choose Save from the contextual menu. |
Figure 1. Data Calculator with data stored in memory
To save any of the results, right-click any of the view controls, and choose Save from the pop-up menu. You can also drag the contents of any view control into any other view control or graph form anywhere else in Source.
The Stats tab gives a statistical summary of the data sets that have been analysed with the Data Calculator. The Reflected Operations tab provides additional data manipulation operations, such as Merge, find Maximum value or multiply two rasters. It allows you to perform customised operations. You can use a plugin to create these operations, which then appear on the list, and can be performed on various data sets.
Data modification tool
The Data modification tool allows you to edit data inputs depending on conditions that you wish to specify. Choose Tools » Data Modification Tool to open the tool, as shown in Figure 2 and carry out the following steps to change the data inputs:
- Click Open under Data loader, which displays the file name in the box below, along with the file on the right;
- Under Condition builder, click Add to add a condition. The input data, conditional operator and condition value can be accessed by hovering and clicking on the relevant area in the condition builder. A drop-down menu appears;
- Click Name to associate a name with the condition;
- Clear can be used to clear the list of conditions if it gets too long and is no longer required;
- Under Rule Executer, you can specify the output for specific aspects of the input. For example, if the condition reaches a certain value, the result will be 0; and
- Click Execute, and the output will be displayed in Result, which can be saved by clicking Save.
Figure 2. Data modification tool
Data unit converter
As its name suggests, this tool converts units (embedded into data files) from one type to another. It is accessible via Tools » Analysis Windows » Data Unit Converter (Figure 3).
Figure 3. Data Unit Converter
Use the converter as follows:
- Drag and drop a data file onto the Source data window in the left side panel. If there are units in the data file, they will appear in the units field below the input window;
- Enter the output/target units in the Units field on the right side (under Converted Data). You can either use the default name (copied from the source data file) or enter a new name; and
- Click Convert. To save the converted file, drag it somewhere else in Source that has a Save as function. This is because you cannot right-click the converted data graph and save it.
If there are no units in the input file, you must force the data converter to assume that there are input units by ticking the Override input units checkbox and entering the "assumed" unit under Units. For example, you have a CSV file containing dates and rainfall, but it does not contain any units. You want an output containing metres per day. Assume the input is in mm/day:
- Drop the input file into the Source data window;
- Tick the Override units checkbox;
- Enter mm.day-1 into the Units field;
- In the converted data window, enter the name for the converted data set;
- In the converted data window units field, enter m.day^-1 (metres per day); and
- Click Convert. The converted data should be scaled down.
You can also scale the converted output to your desired units by ticking the Use converting quantity checkbox. Enter a non-zero value into the Value field and click Convert to scale the output by both the value and the difference in magnitude of the units eg 0.5 mm/h converted to m/h with value 3 ends up being 0.0005 m/h. Note that you must specify both the source value and the target units.
Right clicking in the Converted data dialog will reveal a pop-up menu of options. This enables zooming, panning, dragging, formatting and copying a picture of the output graph to the paste buffer.
Graph Control
Choose Tools » Graph Control to open a charting tool window, which can be used to drag data and display it. Refer to Using the Charting Tool for details.
Mapping Analysis window
The mapping analysis window (accessible via Tools » Analysis Windows » Map...) displays a map of flows and constituent loads per sub-catchment. The relevant catchment variables must have been recorded in the scenario run to be able to view them in the Map Analysis Window. You can perform the following functions in this window:
- To display flows and loads in each sub-catchment, under Map Details, click to open the Runs drop-down menu. Choose a scenario, then the desired Variable and Statistic values from the respective drop-down menus. Move the mouse cursor over a sub-catchment in the map view. A tool-tip appears near the mouse cursor, indicating the variable amount for that sub-catchment per year (Figure 4);
- Flows are reported in ML/d and constituent loads in kg/d;
- You can calculate the difference between two scenario runs by enabling the Subtract checkbox. Carry out the same steps outlined above for the mapping analysis window; and
- Display results per unit area by enabling the Divide by Area checkbox. Flow depth is reported in m/s and constituent load in t/km2/y.
Figure 4. Mapping form with tool-tip showing TSS
| Anchor | ||||
|---|---|---|---|---|
|
A regression test is a project file that has been converted to a format that saves the results for the selected scenarios as 'baseline' results, along with the project. You can then run the regression test in later Source versions, and Source will notify you if the results differ from the baseline results.
Regression tests ensure the following:
- Changes (such as bug fixes) in Source do not introduce additional problems;
- Old projects are compatible with future versions of Source; and
- Correct results are not inadvertently altered because of software changes.
| Info | ||
|---|---|---|
| ||
Note: eWater has a suite of regression tests that are used to check for changes to Source during software development. For each beta software release, there is a list of known changes to these regression tests. To view the regression test changes and a report about the regression test suite for a specific release, see Source Revision History. |
You can create and run local regression tests, and it is good practice to do so for your models before you upgrade to newer versions of Source. Your projects can be included in the eWater Regression Test Suite, please contact eWater if interested.
Running local regression tests
The tools that allow you to create and run regression tests locally are located at Tools » Regression tests. The first steps are to set up the test project, create results to act as the baseline for the regression test and then build the test using the Scenario Test Builder.
Scenario Test Builder
This allows you to create a folder that will contain all files required to run a regression test with your test project.
Use this tool as follows:
- Open or create a project with the scenarios that you want to test
- For each scenario that you wish to test:
- Before running, ensure that you are recording only the parameters relevant to your regression test
- Configure and run your scenario
- Ensure that all the recorded results are giving you the answers you expect
- Open the Scenario Test Builder (Figure 5) using Tools » Regression Tests » Scenario Test Builder
- Click on the ellipsis button (…) and navigate to the folder where you want to save your baseline data and test project for the regression test. Click Select Folder.
- Click Save to save the folder location or Clear to load a different folder.
Once you save, the Scenario Test Builder will create a sub-folder with the same name as your project that contains:
- Sub-folders for each scenario within your project called S01, S02, … etc. Each of these folders contain *.csv files, which are the recorded results for that scenario. These are the baseline files for your regression test comparison, that is, your expected results; and
- A Source project file that the Test Runner loads and runs to compare to the baseline files.
- A Plugins.csv that lists all plugins currently installed in Source, regardless of whether they are used by your test project. When you run your regression test in a later version of Source, you need to have all plugins listed in this file installed. Therefore, if your test project does not use some of the plugins, you can manually edit the Plugins.csv, deleting the rows listing the plugins that are not used. If no plugins are used by your test project, you can delete the Plugins.csv file.
Figure 5. Regression tests, Scenario Test Builder
Create/Edit Regression Test Description file (.xml)
This allows you to document information about the test project and baseline files that were saved by the Scenario Test Builder. It records what functionality each test project is testing and is saved in XML format.
Choose Tools » Regression Tests » Create/Edit Regression Test Description file (.xml) to open the Add Regression Test Project dialog (Figure 6). Once you have entered as much information as you can, click Save.
Figure 6. Regression tests, Add Regression Test Project
Test Runner
This allows you to run regression tests on your previously created test projects. You do this with a more recent version of Source than the one in which you created your test project. For example, you could run a regression test comparing the results generated by the last production release and the most recent beta version.
- Choose Tools » Regression tests » Test Runner to open the dialog shown in Figure 7;
- Click on the ellipsis button to load a folder containing the test project; and
- Click Run to run the scenario.
By default, a fast regression test is run. This loads your test project, runs it, and then compares the result to your baseline results, displaying any errors in the Test Runner window.
Click the Hide Info. button to view fewer details on the errors. The Export All Errors button allows you to save the errors to a file.
The Run slow regression test option is for debugging by Source software developers and , including plugin writers. Compared to a standard regression test, the slow regression test is more thorough, but considerably slower. In a standard regression test, the project is loaded, run, and the results of each scenario compared to baseline. In the slow regression test, the steps of a standard regression test are run multiple times and in various orders times and combined with copy scenario and import scenario to check that all parameters and inputs are loaded and saved correctly, and it verifies everything gets reset that needs to be reset , copied, imported, saved and reset correctly at the beginning of a each run.
Figure 7. Regression tests, Test Runner
Troubleshooting regression tests
If your regression test fails, read the error message(s) and look at the release notes and regression test changes for releases between the version of Source in which you created your test project, and the one in which you ran your regression test. You can also run the regression test in some of these intermediate releases to determine in which version the errors start. If you wish to investigate the errors further, or do not have access to the beta releases, please contact Source support.