Page Comparison

...

The RiverSystem.CommandLine.exe provides a mechanism to run existing projects without the Source interface. It is possible to modify or change the behaviour of the project from the command line, by changing the values of functions. Results can be returned for recorded items or functions within the project.

...

The RiverSystem.CommandLine.exe provides a mechanism to run existing projects without the Source interface. It is possible to modify or change the behaviour of the project from the command line, by changing the values of functions. Results can be returned for recorded items or functions within the project.

RiverSystem.CommandLine.exe is included with Source.

The command line is used where it is impractical to run Source from the main user interface, such as:

For batch runs, where Source is run many times with a range of different parameter values; or
Optimisation scenarios; and
When integrating Source with other models or business systems.

The command line can be used in either a standalone (in-process) mode or in a client/server configuration. When run in client/server mode, the Source project is loaded and run at the server side, with parameters provided by and results returned through the client. In this way, the server can be used for many runs of the same project, without needing to reload the project file each time.

Configuring a scenario from the command line

Any valid Source project can be used with the command line. To modify behaviour of the model, you must configure functions, which can be achieved using the command line.

Figure 1 illustrates the process of manipulating the operating target for a storage from the command line, in this case, manipulating a function, $x. This must first be specified in terms of

...

Functions can be used to manipulate the model in this way, either before the start of the simulation, or before each time-step.

Figure 1. Manipulating functions

Image Modified

The MetaParameter Explorer can be used to attach functions directly to model variables for situations where the Function Editor has not been enabled by default.

Working with the command line

This section describes how to load, run, retrieve results, and modify parameters using the command line.

Info

icon	false

Note: All characters in the command line are case-sensitive.

Loading and running

You can run a project in the following modes:

Standalone mode – the project loads; the simulation runs with a single line command:

Code Block
RiverSystem.CommandLine.exe -p C:\Temp\ExampleProject.rsproj

Client/server mode – one command line runs projects. It does anything except for loading a project until a second command line is run in client mode. In this way, multiple runs can be completed with the server loading the project only once. It also enables the server to run on a different machine to the client. The examples provided assume a client/server mode, illustrating the calls made by the client. They could also work in standalone mode. Refer to Advanced configuration options for two options available in this mode.

Retrieving results

By default, all recorded items are returned, as configured from within the Source application (Running scenarios). Alternatively, you can specify specific items to record by using a command line argument:

Code Block
RiverSystem.CommandLine.exe -m Client -r "Inflow 1/Inflow Volume"

The command after -r refers to the exact name of the recordable item as it appears in Source, ie Node Name/Variable. In this case, the variable, Inflow Volume, of the inflow node will be recorded.

In client/server mode, results are always computed on the server, but returned through the client. By default, results are displayed on the console (in both standalone and client modes), but it is possible to save the results directly to a file using the following command (file name is resultsfilename.res.csv.):

Code Block
RiverSystem.CommandLine.exe -m Client -o resultsfilename.res.csv

Modifying parameters

To modify parameters in the model, they need to be set up as functions. This can be done in one of three ways:

Using the Function Manager directly;
Creating a function and Adding function usages manually; or
Using the Calibration Wizard. Refer to Working with metaparameters for more information.

Info

icon	false

Note: When working with functions in the command line, ensure that functions are referenced by their full name to avoid errors or run failures. The syntax for referencing a function by its full name is $Folder.Function. For example, use:

RiverSystem.CommandLine.exe -v "$Folder1.Function1=10"

Use the -v argument to assign parameter values:

Set the values:

Code Block
RiverSystem.CommandLine.exe -m Client -v "$metaParam=2.5"

Set multiple

...

parameter values:

Code Block
RiverSystem.CommandLine.exe -m Client -v "$metaParam=2.5" -v "$anotherMetaParam=3.0".

Examples

The following set of examples indicate the type of commands used to run projects in the two modes.

Inprocess

The next example runs the project from start to finish and puts all recorded values in the file ‘test.res.csv’.

Code Block

>RiverSystem.CommandLine.exe -p "C:\ProjectDir\test.rsproj" -o test.res.csv
Loading 100%
Meta-Parameters:
+ $inflow
-r "Forest\Catchment for node #2\Quick Flow\Flow"
+ Straight-Through Routing\Default Link #1\Upstream Flow Volume
+ Straight-Through Routing\Default Link #1\Downstream Flow Volume
+ Straight-Through Routing\Default Link #1\Mass Balance
+ Inflow\Inflow 1\Upstream Flow Volume
+ Inflow\Inflow 1\Downstream Flow Volume
+ Inflow\Inflow 1\Inflow Volume
+ Inflow\Inflow 1\Mass Balance
+ Gauge\

...

Server

This command starts a basic server on the same machine as the client:

Code Block
>RiverSystem.CommandLine.exe --mode server

...

In this command, a basic server runs on the same machine as the client. This server will look for project files passed from the client in the directory "C:\ProjectDir":

Code Block
>RiverSystem.CommandLine.exe --mode server --d "C:\ProjectDir"

...

The following command will start up a server, load a project from the directory C:\Users\man25n\Documents\eWater Source Projects using the address specified (net.tcp://192.168.1.5:8080/myWaterService/):

Code Block
>RiverSystem.CommandLine.exe --mode server -d "C:\Users\man25n\Documents\eWater Source Projects" -a "net.tcp://192.168.1.5:8080/myWaterService/"

...

The address consists of the communication protocol (net.tcp) and the name of the end-point. For a machine that runs both the client and server, the address of the end-point must match.

Client

The following command assumes that the server is already running from the previous example:

Code Block
> RiverSystem.CommandLine.exe --mode client --d "C:\ProjectDir"

...

Additional examples of running Source using a DOS batch file, a Python script or an R script can be found here.

...

In the client/server mode, there are two command line windows and RiverSystem.CommandLine.exe is run twice:

XML file: RiverSystem.CommandLine.exe.config – This file contains a section called <system.serviceModel> that can configure hundreds of options for the services and client including transport types, maximum data size, security etc. These are the default options used when running in client or server mode. The location of the server defined in this file, by default, is "localhost" which means the server and client are configured to run on the same machine.
Option A - A project is loaded when the server starts (first command line) and is run in response to a client call (second command line):

Code Block
RiverSystem.CommandLine.exe -p C:\Temp\ExampleProject.rsproj -m Server RiverSystem.CommandLine.exe -m Client

For testing purposes and ad-hoc configuration, two main command line options are also provided to configure the location/type of the server and message time out. When run in this mode the <system.serviceModel> section of the RiverSystem.CommandLine.exe.config file may need to be deleted as creating two servers of the same type is not supported. This means that the net.tcp server on localhost defined in the exe.config file will stop you creating an adhoc server of the same type.
Option B – In the example below, the server starts without loading any project (first command line). The client then specifies the project, which gets loaded onto the server and runs immediately (second command line):

Code Block
RiverSystem.CommandLine.exe -m Server RiverSystem.CommandLine.exe -m Client -p C:\Temp\ExampleProject.rsproj

Option descriptions

Notice in the examples that the command line contains various options, which result in different outputs. The function of these options is provided in Table 1. Note that these are optional extras.

...

Display option commands

...

Function

...

Example

...

-a, --address

...

-a "net.pipe://localhost/PipeReverse" or "net.tcp://localhost:8523/eWater/Services/RiverSystemService"

...

-b, --baseSIUnits

...

--baseSIUnits

...

-c, --recording

...

-c "c:\StateFiles\state.txt"

...

-d, --directory

...

-d "c:\ProjectFiles"

...

-e, --timeout

...

-e 2000

...

-f, --settingFile

...

-f "settings.txt"

...

-l, --loadPlugin

...

-l "plugin.dll"

...

-m, --mode

...

-m Client or -m Server

...

-o, --output

...

-o "output.res.csv" for CSV or -o "output.nc" for NetCDF etc.

...

-p, --project

...

Path to the project, or project name on server and optional scenario within that project, DateTime values to override the default start and end times of the loaded scenario and data source input set to run.

...

-p "projectName.rsproj", or to specify the scenario within the project
-p "c:\somePath\projectName.rsproj;someScenarioName", or to set the run period 
-p "c:\somePath\projectName.rsproj;;1/1/2011;31/12/2011", or to specify a Scenario Input Set
-p "c:\somePath\projectName.rsproj;;;;ScenarioInputSetName" or when using child input sets
-p "c:\somePath\projectName.rsproj;;;;ScenarioInputSetName\ChildInputSetName"

...

-r, --results

...

Which results to return. This can appear multiple times.

Info

icon	false

Note: By default, all recorded results are saved into the specified output file. When this options is used, it only allows you to reduce the number of results saved out. Requested results must be set to record in the project file. To specify the output file name, save the result to a file with a .res.csv extension. Use the column name for the time series.

...

Gauge 2\Upstream Flow Volume
+ Gauge\Gauge 2\Mass Balance
+ Global Expressions\$inflow\Value
Running 100%

Server

This command starts a basic server on the same machine as the client:

Code Block
>RiverSystem.CommandLine.exe --mode server

The service is ready at net.tcp://localhost:8523/eWater/Services/RiverSystemService
Press Enter to stop the service.

In this command, a basic server runs on the same machine as the client. This server will look for project files passed from the client in the directory "C:\ProjectDir":

Code Block
>RiverSystem.CommandLine.exe --mode server --d "C:\ProjectDir"

The service is ready at net.tcp://localhost:8523/eWater/Services/RiverSystemService
Press Enter to stop the service.

The following command will start up a server, load a project from the directory C:\Users\man25n\Documents\eWater Source Projects using the address specified (net.tcp://192.168.1.5:8080/myWaterService/):

Code Block
>RiverSystem.CommandLine.exe --mode server -d "C:\Users\man25n\Documents\eWater Source Projects" -a "net.tcp://192.168.1.5:8080/myWaterService/"

Press Enter to stop the service.

The address consists of the communication protocol (net.tcp) and the name of the end-point. For a machine that runs both the client and server, the address of the end-point must match.

Client

The following command assumes that the server is already running from the previous example:

Code Block
> RiverSystem.CommandLine.exe --mode client --d "C:\ProjectDir"

The service is ready at net.tcp://localhost:8523/eWater/Services/RiverSystemService.

Additional examples of running Source using a DOS batch file, a Python script or an R script can be found here.

Client Server interaction

The service will keep the current state after the client command completes. Projects that are loaded, steps executed and parameters set are still in effect on the server until changed. The client can change this by loading another project or changing state in some other way. This allows for very complicated interactions.

A simple situation is to load a project and run it multiple times with changing values. This would be accomplished by first starting a server

Code Block
> RiverSystem.CommandLine.exe -mode server

Then loading the project in the server from the client in another window while the server is running.

Code Block
> RiverSystem.CommandLine.exe -mode client -p c:\test.rsproj -s 0

NOTE: the "-s 0" in the command above will stop the project from running by default. The project will load and wait, not performing any timesteps ("-s 0")

Next we set some parameter on the loaded project

Code Block
> RiverSystem.CommandLine.exe -mode client -v "$metaParam=2.5" -s 0

Then run the project (running is the default action of the commandline so no extra paramaters need to be specified)

Code Block
> RiverSystem.CommandLine.exe -mode client

Executing the project again with a different value for the parameter can be performed as follows.

Code Block
> RiverSystem.CommandLine.exe -mode client -v "$metaParam=7.5" -s 0 > RiverSystem.CommandLine.exe -mode client

OR just

Code Block
> RiverSystem.CommandLine.exe -mode client -v "$metaParam=7.5"

which will set the parmeter and execute the model (the default operation because no "-s 0" is present to stop the execution)

Anchor
AdvancedConfiguration
AdvancedConfiguration
Advanced configuration options

In the client/server mode, there are two command line windows and RiverSystem.CommandLine.exe is run twice:

XML file: RiverSystem.CommandLine.exe.config – This file contains a section called <system.serviceModel> that can configure hundreds of options for the services and client including transport types, maximum data size, security etc. These are the default options used when running in client or server mode. The location of the server defined in this file, by default, is "localhost" which means the server and client are configured to run on the same machine.
Option A - A project is loaded when the server starts (first command line) and is run in response to a client call (second command line):

Code Block
RiverSystem.CommandLine.exe -p C:\Temp\ExampleProject.rsproj -m Server RiverSystem.CommandLine.exe -m Client

For testing purposes and ad-hoc configuration, two main command line options are also provided to configure the location/type of the server and message time out. When run in this mode the <system.serviceModel> section of the RiverSystem.CommandLine.exe.config file may need to be deleted as creating two servers of the same type is not supported. This means that the net.tcp server on localhost defined in the exe.config file will stop you creating an adhoc server of the same type.
Option B – In the example below, the server starts without loading any project (first command line). The client then specifies the project, which gets loaded onto the server and runs immediately (second command line):

Code Block
RiverSystem.CommandLine.exe -m Server RiverSystem.CommandLine.exe -m Client -p C:\Temp\ExampleProject.rsproj

Option descriptions

Notice in the examples that the command line contains various options, which result in different outputs. The function of these options is provided in Table 1. Note that these are optional extras.

Anchor
CommandLineOptions
CommandLineOptions
Table 1. Command Line Runner Single Analysis (options)

Display option commands

Function

Example

-a, --address

Address for server to create, or client to connect.

-a "net.pipe://localhost/PipeReverse" or "net.tcp://localhost:8523/eWater/Services/RiverSystemService"

-b, --baseSIUnits

Ignore units set within project and output base SI units

--baseSIUnits

-c, -rec, --recording

File to load state from when in client mode or file to save state in when in server mode.

-c "c:\StateFiles\state.txt"

--cp, --configurationproperty

Run configuration property.

--configurationproperty "name=value"

--createsnap

Create a snapshot file for a particular time step

--createsnap "C:\Git\HYDROTESTS\RM_20449_hotstart functionality\test.rsnap"

-d, --directory

Directory in which to look for projects.

-d "c:\ProjectFiles"

-e, --timeout

Operation timeout for WCF service in seconds.

-e 2000

-et, -endtime

The end Data/Time of the scenario run.

--endtime "1/1/2011"

-f, --settingFile

File to merge command-line settings from.

-f "settings.txt"

-?, --help

Show usage help

-?

--is, --inputset

The Scenario Input Set to use for this run.

--inputset "ScenarioInputSetName"

--inputset "ScenarioInputSetName.ChildInputSetName"

--ip, --inputSetProperty

Scenario Input Set property.

These are Scenario input set parameter lines. If you are using child input sets, these are applied after the last child input set. See: Scenario Input Sets

--ip "Functions.Functions.$MyFunction.Expression = 0.25"

--ip "Nodes.InflowNode.Flow =20 ML/d"

-l, --loadPlugin

This option allows a plugin (that the project has been saved with) to be loaded for this run only. The path specified is a relative path to the plugin from the command line. eg. "Plugins\ExampleRainfallRunoffModel.dll"

-l "plugin.dll"

-m, --mode

Application mode Client/Server/InProcess.

-m Client or -m Server

--noPluginFile

Remove all plugins from a server or an InProcess project by the command line.

The plugin(s) necessary for the running project can be loaded individually using the -l command (after --noPluginFile) for the server or InProcess command line.

RiverSystem.CommandLine.exe --mode server –noPluginFile 

RiverSystem.CommandLine.exe -p “Test.rsproj" --noPluginFile -l "plugin.dll"

-o, --output

File to save output into. The extension selects the output format based on the standard TIME I/O formats. A directory of the name specified will be created if the format only supports a single time series.

-o "output.res.csv" for CSV or -o "output.nc" for NetCDF etc.

-p, --project

Path to the project, or project name on server and optional scenario within that project, DateTime values to override the default start and end times of the loaded scenario and data source input set to run.

-p "projectName.rsproj", or to specify the scenario within the project

-md, --ProjectMetadata

Display the metadata for the given project

--ProjectMetadata

-r, --results

Which results to return. This can appear multiple times.

Info

icon	false

Note: By default, all recorded results are saved into the specified output file. When this options is used, it only allows you to reduce the number of results saved out. Requested results must be set to record in the project file. To specify the output file name, save the result to a file with a .res.csv extension. Use the column name for the time series.

-r "someResult" -r "someOtherResult"

--rs

Which recorder set to return. This can be specified multiple times.

By this, we can apply multiple recorders in specified recorder sets to the scenario.

--rs "recordsetname"

--resultsOutputMode

Specifies the way output is created. 3 options are available:

ClientNamedFormat

ServerNamedFormat

NoOutput

"ClientNamedFormat" - Results fetched to client, saved to output file from there, in a format based on output file extension. Default for Client mode. Only valid for Client mode.

"ServerNamedFormat" - Results saved to output file directly from server, in a format based on output file extension. Does not fetch results to client. This "server side" save is more efficient. Default for InProcess mode, since "server side" is local here. Can be used in Client mode, assuming the end user has access to the path provided to the server.

"NoOutput" - Does not save results anywhere, hence does not fetch results either. Can be used where the result data files are ignored. E.g. in tests, or where a plugin gives a different pathway to results.

--resultsOutputMode NoOutput

-o "..\cmdlineResult.res.csv" --resultsOutputMode ServerNamedFormat

-rc, --runconfiguration

Allows you to set the running configuration to be used for the run.

-rc "Flow Calibration"

--scenario

Specify the scenario name

--scenario "New Scenario"

--snap

The snapshot file to apply at the beginning of a run

--snap "C:\some\file\name.rsnap"

--snapshotDate

The time step for which snapshot file has to be created

--snapshotDate 1/1/1990

--starttime

The Start Time for the scenario run

--starttime "1/1/2011"

-s, --step

Number of time-steps to run.

--step 1

-t, --reset

Reset the loaded project.

--reset

-u, --current

Return only the current time step's value for the specified results

--current

...

Set the value of a Function (also called meta-parameter for calibration)

...

-v "$metaparam=25", or -v "$anotherMetaParam=2.3"

...

-y, --saveAs

...

saveAs

Save the loaded project after the run is complete to the specified path and file. The string argument cannot be empty, and the path provided must be an existing directory and have saving permissions for the current user.

-y "C:\ProjectFiles\newProject.rsproj"

--saveAnalytics

This option will generate an XML analytics file for the loaded project. When no file path is specified, the file will be saved next to the project file. Alternatively, a file path can be specified to save the generated analytics file.

If this option is selected along with others in this table, if a file path to save is not specified, it must be the last option chosen.

This option can only be used in InProcess mode.

--saveAnalytics
--saveAnalytics "C:\\foldername\\analyticsfilename.anl.xml"

Anchor
addrunconfig
addrunconfig
Additional run configuration properties

When using a running configuration other than the default Single Analysis, there can be additional configuration properties available. These are handled with the -cp command.

-cp, --ConfigurationProperty

When using River Operations, you can specify the additional properties listed in the table below:

Operations Configuration Property	Data	Example
WarmUpStart	Date Time	-cp "WarmUpStart=1/1/2017"
ReloadOnRun	Boolean	-cp "ReloadOnRun=True"
Today	Date Time	-cp "Today=1/1/2018"
ForecastLength	Positive integer	-cp "ForecastLength=90"
DataUpdatePeriod	Positive integer	-cp "DataUpdatePeriod=15"

For details about these parameters, please see: Operations forecasting

PEST calibration using the command line

PEST (Parameter ESTimation and uncertainty analysis) allows analysis of complex environmental models, and can be used in Source for calibration of catchments. You can use the command line to do calibration and uncertainty in a catchments model. The steps to do the calibration are as follows:

Starting with a Source Catchment project file, use the Calibration Wizard to set up ‘functions’ for the parameters of the rainfall runoff models.
When defining the metaparameters in the Calibration Wizard, the list of Existing Metaparameters will be used in the parameter file for invoking RiverSystem.CommandLine;
Turn on any recorders for the hydrological quantities that will be used in the objective function. In step 4 of the Calibration Wizard, choose Manual optimisationand enable Create functions for use from an external tool;
Save the project;
Set up the RiverSystem.CommandLine input file;
Setup the PEST input and template files;
Prepare a batch file to invoke the commandline and pre-processors; and
Run PEST.

Once the Calibration wizard has been set up, start the command line server (RiverSystem.CommandLine.exe -p projectname.rsproj -m server) and then PEST (pest pestcontrolfile.pst).

Batch running using the command line

Insert excerpt

...

Versions Compared

Old Version 68

New Version Current

Key

Configuring a scenario from the command line

Figure 1. Manipulating functions

Loading and running

Retrieving results

Modifying parameters

Examples

Inprocess

Server

Client

Option descriptions

Server

Client

Client Server interaction

Anchor
AdvancedConfiguration
AdvancedConfiguration
Advanced configuration options

Option descriptions

Anchor
CommandLineOptions
CommandLineOptions
Table 1. Command Line Runner Single Analysis (options)

Anchor
addrunconfig
addrunconfig
Additional run configuration properties

PEST calibration using the command line

Batch running using the command line

Page Comparison

Versions Compared

Old Version 68

New Version Current

Key

Configuring a scenario from the command line

Figure 1. Manipulating functions

Loading and running

Retrieving results

Modifying parameters

Examples

Inprocess

Server

Client

Option descriptions

Server

Client

Client Server interaction

AnchorAdvancedConfigurationAdvancedConfigurationAdvanced configuration options

Option descriptions

AnchorCommandLineOptionsCommandLineOptionsTable 1. Command Line Runner Single Analysis (options)

AnchoraddrunconfigaddrunconfigAdditional run configuration properties

PEST calibration using the command line

Batch running using the command line

Anchor
AdvancedConfiguration
AdvancedConfiguration
Advanced configuration options

Anchor
CommandLineOptions
CommandLineOptions
Table 1. Command Line Runner Single Analysis (options)

Anchor
addrunconfig
addrunconfig
Additional run configuration properties