...
eWater is pleased to announce the latest Production Release of Source version 4.911. Version 4.9 contains the new and updated features developed over the past 6 months since v4.7, along with stability and performance improvements. Version 4.9 also fixes a problem found in two Source models, where the same model would give different results on different computers. Plugin developers should read the details below.
Highlights of Source Version 4.9 include:
| Table of Contents | ||
|---|---|---|
|
Different results on different machines has been fixed!
Technical details for the issue of different results on different machines:
Source uses the Microsoft .NET framework, the problem was caused by 2 functions in the System.Math library. Both System.Math.Pow() and System.Math.Exp() can give different results for some number combinations on different machines. For individual function calls, different results occurred at 0.09% of the time with our randomly generated test data. We believe the issue is from different CPU’s implementing different optimisations to perform the operations as efficiently as possible. The functions are not implemented directly within the Microsoft .NET framework, instead, they are implemented in the Microsoft C run-time libraries which are used by the .NET framework, as well as by a number of other languages including C++. So the nature of the .NET framework, which is an interpreted language, wasn’t part of the issue and the same results would have been obtained if implemented in C++. We have since tested all available System.Math functions and these are the only two we have found that can produce different results on different machines. The vast majority of models don’t see the issue since the differences are very minor and for an individual operation, the difference is only at the last decimal place. However, for very large models with lots of feedback in the model it can snowball into meaningful result differences.
To fix it, we implemented new versions of the two functions that produce consistent results on different machines. The problem has only been detected in 2 Source models, which happen to be 2 of the largest and most complicated. With the fix, we are now getting consistent results for both models, across different machines that were previously giving different results. There will be changes for Plugin developers to use the same functions we are using, instead of the inbuilt System.Math functions when using Pow or Exp. We recommend plugin developers use the new implementations of the functions which are in the TIME.Science.ConsistentMaths class within the TIME assembly.
This was a really difficult problem to isolate and fix. We are VERY happy that we worked it out!!
Model changes to carryover rules over time
Carryover configuration in Resource Assessment can now change over time to assist with calibration.
Map tool improvements
The tool is available under Tools > Analysis Windows > Map... It now works better for constituents and comparing multiple runs.
Calibration performance improvements
The calibration tool is now faster for all models. Models with a large number of parameters will see the greatest benefit.
Data Sources filter
There is now a filter for when looking for a data source or its usage:
Lascam Plugin improvements
The Lascam plugin now includes better value ranges to help determine the appropriate ranges to use when using the calibration tool to calibrate the rainfall-runoff model.
TimeSeries flux for on-farm storages
On-Farm storages at a water user node can now define a Timeseries Flux to assist with calibration. Water added to the On-Farm Storage by the Timeseries Flux functionality only occurs during the Flow Phase. This is in addition to any other Water User Ordering and extractions. The water delivered by the timeseries flux is not actually extracted from the Supply Point.
Charting tool changes
Chart settings have been redone to improve chart stability.
Plugin removal tool
Plugins that need persistence can now include a removal script so plugins can remove themselves from a project when they are not needed or have been added accidentally.
The tool is available in: Tools \ Remove Plugin from Project File.
There is an example provided in the Example Node model plugin.
Minor changes
- The Geographic zoom tool has been fixed and improved.
The run period was constrained when some observed data had been included. We have removed this restriction to allow the model run period to be longer than observed data in the model.
Changes to "Safe Release" functionality for storages with multiple outlets. See: Storage node - OperatingConstraints
- Improvements to recorder sets, See: Configuring Scenarios - Recorder Sets
- Scenario Input Sets, Recorder Sets and Functions can be searched using Ctrl+F
- Run information is now saved in the Results Manager. See: Results Manager - Scenario Run Metadata
- We have improved the performance of editing large constituent models.
- Changes to stop Model Variables breaking when Functions are moved.
- Improvements to handling distribution loss at Supply Points with travel time. See: Supply point node - Distribution Loss
- Improvements to Weirs, including fixing mass balance for constituents with lumped routing.
- Orders can now be redistributed to different accounts at extraction time for use debit accounts. See: Resource assessment - Usage
Community Plugins
Arrange Nodes plugin
You can now use the Arrange Nodes community plugin to rearrange all the schematic nodes of a network. This can be particularly handy if you have built a model starting with a Geographic Scenario. CommunityPlugins \ SourcePlugin.ArrangeNodes.dll
NetLPMonthly Optimiser
A number of changes to the NetLP Monthly Optimiser have been implemented:
- An option to allow the export of model input settings to excel has been added
- Allow adjustment of forecast inflows for 2nd and third replicates
- New recorder "Requirement" for look ahead period in Arc Flows
| Table of Contents | ||
|---|---|---|
|
Version 4.11 includes three significant features:
- You can now combine sub-catchments in a geographic scenario.
- The Calibration tool has been changed from a flow calibration tool to a generic calibration tool.
- You can use the new Environmental flows work with NetLP.
Combine Subcatchments
You can now combine sub-catchments in a geographic scenario:
From selected catchments:
The first catchment selected is deemed to be the Primary Catchment.
At the moment you need to have a single outlet for all of the sub-catchments you are combining. This restriction will be relaxed in future versions.
At Selected Node Boundaries:
You can select a single node and combine all of the catchments upstream of it.
Calibration tool enhancements
The calibration tool has undergone significant enhancements. It has been changed from a flow calibration tool to a generic calibration tool. You can now specify any recordable output (including functions) to be used as part of the objective function in the calibration wizard. The independent variable can now be a Data Source, rather than a separately loaded text file.
We still have more work to do in allowing any parameters to be modified. We have enabled users to calibrate constituents as a first step.
Environmental Flows with NetLP
The new Environmental Flow node and Environmental Flow Manager now work with NetLP ordering. To determine the "marginal" difference in orders that are required by the Environmental Flow Node, NetLP determines probable flows throughout the system prior to determining environmental flow orders. The model does the order phase twice - firstly Expected Flows without initiating new Environmental Flow orders are determined. These flows are then provided to the environmental Flow Nodes to perform their ordering calculations. NetLP is then given the environmental flow demands as well as all of the other demands and works out the most efficient way to deliver the orders.
Minor changes
- Improved generation of orders at Storages and Weirs. Source now allows the user to specify a factor to smooth the rate at which downstream storages orders water from upstream.
- Off allocation had an option called "Equalise Shares", which wasn't used by any of the jurisdictions, and has now been removed.
- climate Data import tool improvements, including the ability to extend the duration of data
- zoom all the time with the scroll wheel for Catchment Scenarios
- a fix for projects being corrupted by windows (affected a small number of users)
- modelled variables are now more stable when moving links
- optional improvement of ordering using Operating Targets with Rules-based orders at storages
- resolution of infinite loops
- Insight now produces a cache of model run results
- addressed some charting issues.
Infinite loops removed from Off Allocation and the Environmental Flow Manager
We have modified Off Allocation and Environmental Flow Manager distribution logic to remove the infinite loops that could occur in some models.
Insight changes
Insight now produces a file which is a cache of model run results. This means that when you run an insight optimisation and then decide to increase the number of generations of the optimisation, you can continue the extended optimisation from the end of the previous optimisation. This also means you can stop an insight run and restart it without wasting a large amount of time.
Chart changes
We have resolved a number of issues identified with charts and the mass balance tool since their redevelopment.
Chart aggregation can now ignore null values. This allows for the aggregation of seasonal data to an annual time series. e.g. you can now generate a series of the total summer inflows each year.
Community Plugin Changes
- Blackwater plugin Initial conditions can now be specified as varying according to elevation
Plugin Developers
ScenarioRunFinalise now happens on all plugins. If there was an exception in one plugin, it previously wouldn't run ScenarioRunFinalise for the remaining plugins when there were multiple plugins.
We are currently on .net 4.6.2, which hasn't changed from Source 4.7. We are planning on moving to C# 7, Visual Studio 2019 and .NET 4.8.0 soon, so you will need to make sure you have updated Visual Studio to continue developing plugins in the beta versions. We also intend to move to .NET 5 in 2020. .NET 5 will enable the Source command line and service interface to run on Linux. Note the section above on the fix for different results from the same model on different machines.
Results and Configuration Changes
Some results have changed between Source 4.7 9 and Source 4.911. Result changes are mostly around Annual Accounting, Weirs and specific ordering settings. The eWater development team maintains a detailed system to track when results vary between different version of Source. The details at the following link will help you work out why the results have changed, and any alterations you may need to make to your model configuration. Details of result changesweirs and improvements to unit validation: 4.11 Result Changes from 4.79