| Excerpt | |||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
You can undertake thousands of different model runs, each with varying time series inputs. This can be accomplished by running multiple instances on a single server (locally - one per CPU), or over a cluster of servers (across a network). To undertake a batch run, you either:
The parameters required to configure a local batch run are summarised in Table 2.
Table 2. Parameters for Local Run Settings
|
|
The example contains a number of files and folders and a small example project that was built in 3.6.3beta.
An example can be downloaded here: Example Batch Run scripts using Source 3.6.3beta
Long runs in Source of about 10,000 years
Some organisations like to use Source to do runs of about 10,000 years. Here are some tips from Chas Egan in Qld:
- Record just the bare minimum number of things that you need, like just 15 things.
- Result streaming is a must, otherwise, memory use will end your run really quickly. Even with result streaming on we find that the number of runs we can put on a machine is limited by memory (not CPU cores). So be aware of the memory footprint before launching lots of replicates.
- Run from 01/07/0002 to 30/06/9998. Source is limited by the date ranges available in DateTime, year 0000 to year 9999. However, you can’t actually set the simulation dates to the full range because Source needs to look at DateTime values a bit beyond either end of the simulation for things like Resource Assessment.
Just because your model runs over a historical length, doesn’t mean it will run over 10k years.
- MAKE SURE YOU TURN OFF PARALLEL PROCESSING. There’s a bug in there which shows up in long runs.
- Make sure your storage dimension curves, spillways, valves, splitter effluent tables, all have nice shapes. Large numbers or step functions in those tables will likely cause problems over a long run.
- If you have any function demands, wrap them in “max(0, …)” to prevent small floating-point errors from making negative demands and killing your runs.
- We still get random failures of some of our replicates. And therefore we design our postprocessing to work fine if, e.g., only 98 of our 100 replicates survive.
We have some stochastic work coming up for the Border, which takes 10x as long as the Granite Belt, and for that we have decided to reshape our solution to use shorter runs (say 1000 years) and more replicates. I highly recommend looking at that if it works for you. Not only will it let you share the processing to more machines, but it will also mean if/when your runs fail, you’ll only have lost a day (rather than a week).
Changing Temp Data location
You can also change the path the temp data is stored.
By default they are stored in:
Program Files (settings, logs): C:\Users\<Username>\AppData\Roaming\eWater\Source\
Temp Files (projects, plugins, run info): C:\Users\<Username>\AppData\Local\eWater\Source\
These can be changed through teh user interface (Edit \ Application Settings... \ Application User Data). If you would like to do it via the commandline, you can add in registry settings using the following commands:
In a subfolder of the working directory:
reg add "HKCU\Software\eWater\Options\Source 5.2.0\RiverSystem\Options\RiverSystemOptions" /v "LocalBaseSourcePath" /d ".\Temp" /f
reg add "HKCU\Software\eWater\Options\Source 5.2.0\RiverSystem\Options\RiverSystemOptions" /v "BaseSourcePath" /d ".\Temp" /f