Batch running using the command line
- asatheesh (Unlicensed)
- Geoff Davis
You can undertake thousands of different model runs, each with varying time series inputs. This can be accomplished by running multiple instances on a single server (locally - one per CPU), or over a cluster of servers (across a network).
To undertake a batch run, you either:
- Configure the script settings directly;
- Pass a .cmd settings file to the script; or
- Run a simple GUI for editing configurations and running them.
The parameters required to configure a local batch run are summarised in Table 2.
Table 2. Parameters for Local Run Settings
| Parameter | Description | Example |
|---|---|---|
| Server.Exe | Location of the RiverSystem.CommandLine.exe that will be run for the server. This should be the same as ClientExe when running locally or a path than can be reached from the server. | \\Server\Source\RiverSystem.Commandline.exe |
| ProjectFile | The template project file that will be run on the server. The file location (including the directory tree) will be copied multiple times and merged with the directories from InputDir. | \ExampleMasterProject\Batch.rsproj |
| ProjectDestinationDir | The working location that should be accessible from the Server and client machines. Files from the InputDir will be copied to this location to change input files. | \\Server\Volume_1\ExampleProjectLoadLocation |
| InputDir | Contains a directory for each run. Files that change should be in these directories, and are copied to the ProjectDestinationDir during a run. | |
| OutputDir | The location that output files from runs are saved in the form Result_nameOfInputDir. | |
| NumServers | The number of servers that will be started for the job to be distributed over. |
The example contains a number of files and folders and a small example project that was built in 3.6.3beta.
An example can be downloaded here: Example Batch Run scripts using Source 3.6.3beta
Long runs in Source of about 10,000 years
Some organisations like to use Source to do runs of about 10,000 years. Here are some tips from Chas Egan in Qld:
- Record just the bare minimum number of things that you need, like just 15 things.
- Result streaming is a must, otherwise, memory use will end your run really quickly. Even with result streaming on we find that the number of runs we can put on a machine is limited by memory (not CPU cores). So be aware of the memory footprint before launching lots of replicates.
- Run from 01/07/0002 to 30/06/9998. Source is limited by the date ranges available in DateTime, year 0000 to year 9999. However, you can’t actually set the simulation dates to the full range because Source needs to look at DateTime values a bit beyond either end of the simulation for things like Resource Assessment.
Just because your model runs over a historical length, doesn’t mean it will run over 10k years.
- MAKE SURE YOU TURN OFF PARALLEL PROCESSING. There’s a bug in there which shows up in long runs.
- Make sure your storage dimension curves, spillways, valves, splitter effluent tables, all have nice shapes. Large numbers or step functions in those tables will likely cause problems over a long run.
- If you have any function demands, wrap them in “max(0, …)” to prevent small floating-point errors from making negative demands and killing your runs.
- We still get random failures of some of our replicates. And therefore we design our postprocessing to work fine if, e.g., only 98 of our 100 replicates survive.
We have some stochastic work coming up for the Border, which takes 10x as long as the Granite Belt, and for that we have decided to reshape our solution to use shorter runs (say 1000 years) and more replicates. I highly recommend looking at that if it works for you. Not only will it let you share the processing to more machines, but it will also mean if/when your runs fail, you’ll only have lost a day (rather than a week).
Changing Temp Data location
You can also change the path the temp data is stored.
By default they are stored in:
Program Files (settings, logs): C:\Users\<Username>\AppData\Roaming\eWater\Source\
Temp Files (projects, plugins, run info): C:\Users\<Username>\AppData\Local\eWater\Source\
These can be changed through teh user interface (Edit \ Application Settings... \ Application User Data). If you would like to do it via the commandline, you can add in registry settings using the following commands:
In a subfolder of the working directory:
reg add "HKCU\Software\eWater\Options\Source 5.2.0\RiverSystem\Options\RiverSystemOptions" /v "LocalBaseSourcePath" /d ".\Temp" /f
reg add "HKCU\Software\eWater\Options\Source 5.2.0\RiverSystem\Options\RiverSystemOptions" /v "BaseSourcePath" /d ".\Temp" /f