Overview
The SubSource plugin is developed to run a second Source model at different points during the main model run. This plugin is typically used to forecast water allocations or integrate different time steps in the operation model.
The basic concept can be expressed , as shown in Figure 1, can be explain as follows:
The main model (blue)
hassteps through the simulation period, using some information from
Subthe sub-
Modelmodel (yellow) to calculate water allocations.
At each call from the main model,
Subsub-
Modelmodel (yellow) simulates the scenario such as a worst dry case until the end of the water year (or a defined end time by the user) to estimate harvestable inflows and losses for the main model.
Figure 1 The basic concept of SubSource
Although the plugin is was originally developed for the a specific project, but it is flexible for it has since been extended to be a flexible generic tool and may be used to address different needs in more complex water system conditionmodels. The main model and Subsub-Model model scenarios can be from defined in the same Source project file or from different Source project fileseparate Source project files. It is also possible to use the same scenario for both the main model and sub-model using different running configurations, input sets, and recorder sets. The developed interface allows the user to choose different parameter values for their requirements. The models can use the catchment mode or the network modelmode. The user Users can design their own data mapping mappings between the main model and Subsub-Model. The plugin is convenient to be applied for other projects / purpose.
Figure 1 The basic concept of SubSource
Access the functions of SubSource plugin and the basic proceduremodel.
| Table of Contents | ||
|---|---|---|
|
Basic procedure for use of the SubSource plugin
The basic procedure
for the
use of the SubSource plugin
is described below:
- Step 1 Create the Source model using SubSource function
Create a new project
Load the SubSource plugin from CommunityPlugins\SourcePlugin.SubSource.dll
in the Plugin Manager.
- Create a new project (or alternatively open an existing project to add the plugin to)
- Step 2 Create models
Create a main model (scenario) in the current project (
see Figure 2(1))
Save the project
Create a sub model (scenario) in the current project (
see Figure 2(2)) or another Source project file
Save the project.
- Step 3 Prepare the
- mapping between the main model and
- sub-
- model.
Make the Main Model (scenario) active
Select Sub-model analysis
from the Analysis Type dropdown list (
see Figure 2(3))
Click
Configure (
see Figure 2(4)) on the Simulation toolbar to display the Sub-Model Analysis Configuration
dialog (Figure 3)
Set up Sub-Model Analysis
configuration and mapping.
- Step 4 Save the Source project file(s) after the Sub-model analysis
- has been set up or modified.
Step 5 Run Main Model.
Figure 2 Access the functions of SubSource Plugin
Figure 3 Sub-Model Analysis Configuration Interface
Configure Sub-Model Analysis
One additional section (i.e. Sub-Model) under Configuration tab was developed to utilize SubSource functions (Figure 3).
In the Sub-Model section, there are two kinds of functions, (a) Main model to sub-model mapping and (b) Set up running parameters for Sub-Model. The functions developed in this section are described below. In this document, the sub model represents the source scenario of Sub-Model with the original condition while Sub-Model is the selected scenario for additional setting up about Sub-Model Analysis or already with additional setting up.toc- Although the latest version of the SubSource plugin can automatically save the project to a temporary location prior to run, it is recommended to save the project to make sure changes to the the sub-model are applied.
- Step 5 Run the main model using the defined Sub-model analysis configuration.
Figure 2 SubSource plugin components
Sub-Model Analysis configuration
The Sub-Model Analysis running configuration is based on the Single Analysis running configuration with one additional section, Sub-Model (see Figure 3).
The Sub-Model section allows the configuration of:
- the mapping between the main model and sub-model; and
- the parameters for the sub-model run;
Figure 3 Sub-Model Analysis Configuration dialog
Main model to sub-model mapping
Define the multiple sub-model mappings
Clicking on Edit button under the frame of Main model to sub-model mapping (Figure 3), or clicking on the tools menu Tools » Sub-model Manning Manager ( Figure 4) , will open the Sub-model Mapping Manager interface (Figure 5)
Figure 4 Tool for Sub-model Mapping Manager
Figure 5 Sub-model Mapping Manager interface
The left box in Figure 5 can be used to display the list of multiple names and it allows the user to manage this list of Sub-model Mappings:
Right click on any empty place in the left box of Figure 5 will display the Add Mapping button shown as Figure 6. Click on the Add Mapping will display all parameters, which are associated with the definition of a Sub-model Mapping, in the right box with empty value.
Figure 6 Add Mapping
Right click on any name in the left box of Figure 5 will display the Delete button and click on this Delete will delete this name (, or a definition of Sub-model Mapping,) from the modelling.
Click on any name in the left box of Figure 5 will display all parameters values associated with the definition of this selected Sub-model Mapping in the right box. The user can display or edit this sub model mapping definition.
The details of all parameters associated with a definition of one Sub-model Mapping are given below:
Name: the unique name of the current Sub-model Mapping.
Use Current Project: this tick box defines which Source project file is used to select a scenario as Sub-Model. If it is ticked, the interface will load all scenarios from the current project into Scenario dropdown list for the option. If it is not ticked, Project Path parameter will be activated to define the source project file.
Project Path: this parameter allows the user to choose a Source project file (other than the current project file) as the source of Sub-Model. It is disabled when Use Current Project is ticked.
Loading…: this progress bar will display the progress information loading all scenarios in the project file to Scenario dropdown list ( talk below).
In Sub-mode Running Configuration frame in Figure 5:
Scenario: the user can select one relevant scenario from this list as Sub-Model. All scenarios in the used Source project file are available for choice.
Sub-Model not only can be selected from same or different project where the main model comes from, but also can use the same model (scenario) as that for the main model.
Override Run Configuration: this tick box allows the user to change the default of Running Configuration names for the selected sub model. When it is ticked, the Configuration dropdown list will be activated.
The Running Configuration names are defined by Running Configuration Management from Scenario Options menu.
Configuration dropdown list: It provides the available options of Running Configuration names (such as single analysis, linked analysis, custom 1) for Sub-Model. The model will run Sub-Model using selected Running Configuration name from this parameter and ignore default one in the sub model / original scenario. It is only available when the Override Run Configuration tick box is ticked.
Scenario Input Set: this parameter allows to select different existing Scenario Input Sets for Sub-Model run.
From Variable Mapping From Main Model to Sub-Model frame, the user can use it to set up data transfer from the main model to Sub-Model and it is happed when Sub-Model is triggered by the main mode to start running:
Main Model Value: it is used to define a source value in the main model. This source can be a specific value, a data in data source or a function. It will be signed to the objective function in Sub Model when the main model triggers running of Sub-Model.
Sub-Model Function: it is used to specify a function (existed in Sub-Model). This function will adopt the value from Main Model Value when Sub-Model starts running. Although all functions in Sub-Model are available options in its dropdown list, but the user needs to select a corresponding function to transfer the data.
Note that:
The time for the main model is the end of last time step while the time trigged to run the Sub-Model is the start of the time step. For example, the main model (daily time step) trigged Sub-Model (Monthly time step) to run from 1 September 2000. The time for the main model is the end time step of 31 August 2000, and the condition (data) at this time step will be used to setup the initial condition of Sub-Model. The time for Sub-Model is the start of 1 September 2000. This time will be used to receive the initial data from the main model. These concepts are practically for Time of Evaluation.
For the function in this frame, the user can add as many as they need.
Each function only allows one value to be used for the data transfer. If the time series is used, the last value will be taken.
All initial conditions of the water system in Sub-Model need the setting up such as storages' initial volume, water account balance in RAS. These initial values often are passed though the functions from the man model to Sub-Model.
“Run Initialisation" and "Start Of Run” of Time of Evaluation are specifically developed for this plugin:
"Run Initialisation" functions are functions which are used when initialising the run and network elements. Functions which are set to this Time of Evaluation cannot use Model Variables
"Start Of Run" will be evaluated after initialisation (reset), and can be used to prime functions and Model Variables with calculated initial values
From Variable Mapping From Sub-Model to Main Model frame, the user ca use it to set up data transfer from Sub-Model model to the main:
Sub-Model Recorder: it is a recorder in Sub-Model to save desired running results as a data source and this source value(s) will be signed to an objective function in the main model when Sub-Model completed a running process.
Main Model Function: it can be used to specify a function (existing in the main model), which will adopt the value(s) from Sub-Model Recorder when Sub-Model completed a run process. Again, the user needs to select a relevant function to transfer the data.
Note that:
The user only needs to save the recorders, which are necessary for data transfer, in Sub-Model. This can reduce the model running time and the computer memory.
The principle of above notes for Variable Mapping From Main Model to Sub-Model is also applied for this frame.
Select a mapping
Th user selected mapping in Figure 5 will be displayed on the dropdown list of Mapping such as “my Sub Model Mapping” in Figure 3, and this mapping will be used during the model running. The user also can directly select the existed /pre-defined mapping items from the dropdown list of Mapping.
Note that the user must remember to save the Source project file before you run the mode because the setting up is functionalized only after the project is saved.
Set up running parameters for Sub-Model
From the lower section of the interface of Sub-Model Analysis Configuration (Figure 3), the parameters, which are developed to meet different needs in Sub-Model run, are described below:
Time Step: this parameter provides three options (i.e. From Main Model, From Sub-Model and Custom Time Step) to set up simulation time step for Sub-Model.
From Main Model: the time step used in Sub-Model will be same as one defined in the main model.
From Sub-Model: the time step used in Sub Model is using the one in the sub model/original scenario.
Custom Time Step: this option will display an additional dropdown list for the user to select the required time step. Should note that the user needs to consider the time step used in Main Model for a reasonable matched time step.
Start Time: this parameter provides four options, called From Main Model, From Sub-Model, Custom Start Time and Main Model Trigger Time, to set up the simulation start time for Sub-Model. The first three options are self-explained, and their mechanisms are same as those in Time Step. Main Model Trigger Time option is using the time when the main model triggered Sub-Model as the simulation start time of Sub-Model run, and this is a default option. For example, the main model triggered Sub-Model on the time start of 1st September 2000, Sub-Model will simulate the water process from 1st September 2000.
Run Period: this parameter provides four options (i.e. From Sub-Model, End of Water Year, Number of Time Steps and Custom End Time) to set up the simulation period for Sub-Model
From Sub-Model: the model run period will be adopted from the sub model/source scenario. The run end of time could be different to one defined in the source scenario if the selected start time is not from the source scenario.
End of Water Year: the run period will be from the simulation start time to the end of water year. This option will display the additional item (at the bottom of the interface) to define the water year.
Number of Time Steps: it defines how many time steps (such as months) and Sub-Model will run the simulation in this amount of time steps.
Custom End Time: it allows the user to specify a date when the simulation of Sub-Model will be stopped. This option will display an additional text box to enter a date as the simulation end time.
Run Frequency: this parameter provides five options (i.e. Start of Week, Start of Month, Start of Water Year, Number of Time Steps and Custom Trigger) to specify the run frequency for Sub-Model
Start of Week: the action to run Sub-Model will be repeated at the beginning of each week.
Start of Month: the action to run Sub-Model will be repeated at the beginning of each month.
Start of Water Year: the action to run Sub-Model will be repeated at the beginning of each water year.
Number of Time Steps: the action to run Sub-Model will be repeated after a certain time interval defined by this parameter.
Custom Trigger: it defines a Boolean function to decide whether Sub-Model will be triggered by the main model to start run. The function can be a system conditions such as the water level at one reservoir reached to a certain level (and then Sub-Model will start to run).
Water Year: this parameter is linked to options of End of Water Year and Start of Water Year. It is used to specify the start day and month of the water year.
The user needs to consider above parameters together to define their values in a systematic approach.
Note that the user must save the Source project file because the setting up is functionalized only after the project is saved. Otherwise, Sub-Model will be using the previously saved version of the model.
The models also need a proper design and debugging, especially for data transfer between main model and sub model.
Select the mapping to use for passing values between the main and sub-models. Click Edit to open the Sub-model Mapping Manager to modify or create new mappings.
Sub-model run parameters
The options in the lower section of the Configuration tab (Figure 3) are for controlling the sub-model run. In this section, "Main Model" is used to denote values / settings taken from the current state of the main model when the sub-model run is triggered, while "Sub-Model" denotes the original values / settings defined in the sub-model running configuration. Using "Main Model" will update the running configuration of the sub-model prior to starting the sub-model run.
Time Step: this parameter provides three options (i.e. From Main Model, From Sub-Model and Custom Time Step) to set the simulation time step for the sub-model run:
From Main Model: the time step used in the sub-model run will be the same as the one defined in the Main Model section of the Configuration tab.
From Sub-Model: the time step used in the sub-model run will not be modified, using the the original running configuration setting.
Custom Time Step: this option will display an additional dropdown list for the user to select the required time step. Users need to consider the time step used in the main model and data sources in the sub-model for a reasonable matched time step.
Start Time: this parameter provides four options, called From Main Model, From Sub-Model, Custom Start Time and Main Model Trigger Time, to set up the simulation start time for sub-model run. The first three options are self-explanatory, and their mechanisms are same as those in Time Step. Specifying the Main Model Trigger Time option, the default option, will use the time when the main model triggered the sub-model run as the simulation start time of the sub-model run. For example, if the main model triggered a sub-model run on the 1st September 2000, the start time of the sub-model run will be modified to simulate the water process from 1st September 2000.
Run Period: this parameter provides four options (i.e. From Sub-Model, End of Water Year, Number of Time Steps and Custom End Time) to set up the simulation period for the sub-model run:
From Sub-Model: the sub-model run period will be adopted from the sub-model's running configuration. The run end time could be different to the one defined in the sub-model's running configuration if the selected Start Time is not also set to From Sub-Model.
End of Water Year: the run period will be from the simulation start time to the end of the water year. This option will display an additional entry field (at the bottom of the tab) to define the water year.
Number of Time Steps: the sub-model run period will be a fixed number of time steps from the simulation start time. This option will display an additional entry field for the number of timesteps to run.
Custom End Time: allows the user to specify the end time to be used for the sub-model run. This option will display an additional entry field to enter a date as the simulation end time.
Run Frequency: this parameter provides five options (i.e. Start of Week, Start of Month, Start of Water Year, Number of Time Steps and Custom Trigger) to specify when the sub-model will be triggered during the main model run:
Start of Week: the sub-model run will be triggered at the beginning of each week.
Start of Month: the sub-model run will be triggered at the beginning of each month.
Start of Water Year: the sub-model run will be triggered at the beginning of each water year. This option will display an additional entry field (at the bottom of the tab) to define the water year.
Number of Time Steps: the sub-model run will be triggered after a recurring number of timesteps. This option will display an additional entry field for the number of timesteps.
Custom Trigger: the sub-model run will be triggered when the specified Boolean function evaluates to True. The function can use system conditions (such as the water level at a reservoir reaching a certain level or the flow rate at a node exceeding a threshold) to determine whether to trigger the sub-model run. The function time of evaluation must be prior to when the sub-model run will be triggered, such as End of Timestep. This option will display an additional entry field for selecting the function.
Water Year: this parameter is linked to options of End of Water Year and Start of Water Year. It is used to specify the start day and month of the water year.
The user needs to consider the above parameters together to define their values in a systematic approach.
Sub-model Mapping Manager
The Sub-model Mapping Manager dialog (Figure 5) is used for defining mappings between main and sub-models. A Source scenario may have multiple mappings defined and each mapping may be used by multiple Sub-Model Analysis running configurations within the scenario. Mappings are defined in the main model scenario. No mappings settings are required in the sub-model scenario (other than the required functions and recorders).
The Sub-model Mapping Manager dialog (Figure 5) may be accessed either by clicking the Edit button next to the Main model to sub-model mapping setting in the Sub-Model Analysis Configuration dialog (Figure 3), or from the Source Tools menu, Tools » Sub-model Mapping Manager (Figure 4).
Figure 4 Access Sub-model Mapping Manager from the Tools menu
Figure 5 Sub-model Mapping Manager dialog
Sub-model mapping management
The left pane of the dialog (Figure 5) lists the sub-model mappings defined in the scenario, allowing for adding, deleting and editing of mappings:
- To add a new mapping, either click the Add Mapping toolbar button or right click an empty location in the list and click Add Mapping from the context menu. A new mapping will be created and selected, displaying parameters in the right pane.
- To delete a mapping, select the mapping from the list and either click the Delete toolbar button or right click the mapping entry and click Delete from the context menu. The mapping will be deleted from the scenario. Any running configurations which referenced the mapping will need to be edited to specify a different mapping.
- To display and edit a mapping click / select the mapping from the list to display the parameters in the right pane.
Sub-model mapping parameters
Sub-model mapping parameters are displayed in the right pane (Figure 5) when a mapping is selected. The details of all parameters defined in a sub-model mapping are as follows:
Mapping details
Name: the unique name of the current sub-model mapping.
Use Current Project: this option controls whether to use the current Source project file or a different project file for selecting the sub-model scenario. Ticking this option will load all scenarios from the current project into the Scenario dropdown list. Unticking this option will enable the Project Path field for specifying a different Source project file.
Project Path: this parameter allows the user to choose a Source project file (other than the current project file) for selecting the sub-model scenario. It is disabled when Use Current Project is checked.
Sub-model Running Configuration
Scenario: select the scenario from the dropdown list to use as the sub-model. All scenarios in the specified Source project file are available for choice.
The sub-model scenario may be either another scenario in the current project, a scenario in another Source project file, or the same scenario as the main model in the current project.Override Run Configuration: this option controls whether to use the default running configuration defined in the selected scenario or to specify another running configuration from the Configuration drop down list below. When ticked, the Configuration dropdown list will be enabled.
Configuration: select the running configuration from the dropdown list to use for the sub-model run. Only Single analysis running configurations are supported for sub-model runs. The listed running configurations are those defined within the Scenario Options » Running Configuration Management settings for the selected sub-model Scenario. The parameter is only enabled if Override Run Configuration option is ticked.
Scenario Input Set: Displays the Scenario Input Set for which will be used for the sub-model run. Currently cannot be modified.
Variable Mapping From Main Model to Sub-Model
In this list the user can define data transferred from the main model to the sub-model at the time the sub-model run is triggered:
Main Model Value: specifies the source of the value in the main model; a specific value, a data source timeseries, or a function.
Sub-Model Function: specifies the function in the sub-model scenario which the main model value will be set to. All functions in the sub-model scenario are listed in the drop down list which also supports filtering and auto-complete if the path to the function is typed. The text of the function will be overwritten with the value from the main model. Use of the function in the sub-model will need to be setup as normal for functions.
To add a new mapping entry, double click the empty row at the bottom of the list. To delete a mapping entry, click the entry in the list to select it and press the Delete key on your keyboard.
Note that:
- Each function only allows a single value to be passed from the main model to sub-model. Arrays, multi-value variables or modelled variables using date ranges are not supported. If a time series is used, the value for the current main model time step is used.
Generally, all initial conditions of the water system in the sub-model need to be set from main model values, such as storage initial volumes, water account balances in RAS. These initial values should be defined using functions in the sub-model and corresponding mapping entries added in the mapping defined in the main model.
Run Initialisation and Start Of Run function Time of Evaluations were specifically added to Source for use by SubSource and should the the Time of Evaluation used for functions within this list:
Run Initialisation functions are evaluated and used when initialising the run and network elements.
Start Of Run functions are evaluated and used after initialisation (reset), and can be used to prime functions and modelled variables with calculated initial values
Variable Mapping From Sub-Model to Main Model
In this list the can define data transferred from the sub-model to the main model after the completion of a sub-model run:
Sub-Model Recorder: specifies a recorder in the sub-model scenario. The text box supports auto-complete for searching for a recorder.
Main Model Function: specifies the function in the main model scenario which the value from the sub-model recorder will be set to. All functions in the main model scenario are listed in the drop down list which also supports filtering and auto-complete if the path to the function is typed. The text of the function will be overwritten with the value from the sub-model recorder. Use of the function in the main model will need to be setup as normal for functions. These function will appear to be constant functions to Source so it is crucial that they have Force Evaluate set to true for their values to be updated and used by the main model.
To add a new mapping entry, double click the empty row at the bottom of the list. To delete a mapping entry, click the entry in the list to select it and press the Delete key on your keyboard.
Note that:
The sub-model only needs to have the recorders for the mappings enabled. All other recorders should be disabled in order to reduce the sub-model running time and memory use. It is recommended to use recorder sets for this purpose.
- Only the last recorded value of the sub-model recorder is assigned to the main model function. Arrays, multi-value variables or modelled variables using date ranges are not supported. If multiple values are required then functions should be defined in the sub-model which use date ranges and modelled variables to provide values from previous timesteps and these functions recorded.
Running the analysis and general guidelines
The sub-model analysis run is run from the main model in the same way as other analysis types in Source. The sub-model run will be run when triggered from the main model using the Source Service in a separate process.
Points to consider when designing the main and sub-models:
- When triggered, the sub-model run will run between the End of Timestep and Start of Timestep simulation phases. Variable mappings from the main model to the sub-model will thus use values from the end of the time step before any new values are applied (functions evaluated at Start of Timestep and timeseries values played onto a model). Similarly, variable mappings from sub-model to main model will have values applied to the main model before Start of Timestep so that any functions evaluated at Start of Timestep in the main model can use the values from the sub-model run.
- For example, the main model (daily time step) triggered the sub-model (monthly time step) to run from 1 September 2000. The time for the main model is the end time step of 31 August 2000, and the condition (data) at the end of this time step will be used to setup the initial condition of the sub-model. The time for the sub-model is the start of 1 September 2000.
- See simulation phases and function times of evaluation for further details of timings.
- Sub-model run results are not recorded in the main model results. It can therefore be difficult to debug what is happening in the sub-model runs. To help with debugging a few approaches may be used:
- Auto export results scenario option in the sub-model scenario may be enabled to export run results to a results file and later loaded and viewed in Results Manager. Note that both the Append Run Number and Append Date and Time options should be enabled to prevent subsequent run results overwriting previous run files which may have the same name.
- Auto export simulation log scenario option in the sub-model may similarly be used to export results summaries and log messages from the sub-model runs. It may be useful to use this together with function logging.
- Record functions and data sources used in the main model to sub-model mappings.
- Set up and record additional functions in the sub-model and define functions in the main model to receive these values which and subsequently recorded. Set Force Evaluate to true to make sure the functions are updated correctly.
- It is recommended to create a Single analysis configuration in the sub-model scenario for exclusive use by the sub-model run. A recorder set should be created for this configuration and the Current recorder tree recorder set disabled. This will help prevent runtime and configuration errors due to changes to recorders for doing other types of analysis runs in the scenario.
- When using the same project and scenario as both main and sub-models use of separate running configurations and recorder sets is crucial. It is also important to understand that the functions used in the sub-model run will not contain values from the sub-model if they are recorded during the main model run. The models should be treated as separate models as much as possible to avoid confusion.
- If using the same project file for main and sub-model scenarios, the project file will be saved to a temporary location prior to run so as to get the latest changes made to the project. The project is not required to be saved prior to changing mapping settings as meta-data will be loaded from the current project rather than using the Source Service.
- If using separate project files for the main and sub-model scenarios, the sub-model project file must be saved prior to run if it has being modified in a separate instance of Source. If it is not saved the previous version of project file will be used for the sub-model runs. Similarly, the sub-model project file will need to be saved after any recorder or function name changes in order for these changes to be used in the main model mapping configuration.
- The latest version of Source and the SubSource plugin can upgrade projects created with previous version of the SubSource plugin which previously didn't have the Sub-model Mapping manager.
- If using a new version of Source and/or the SubSource plugin it is recommended to upgrade and save the project file(s) prior to any runs. If it is not upgraded prior to run then the project file(s) will be upgraded each time the Source Service loads the sub-model project file which could take a long time for some projects.
- Although the SubSource plugin is not specifically used by the sub-model project, it is recommended to add SubSource (and all other plugins used in the main model project) to the sub-model project. All plugins present in Source when running the main model will be loaded by the Source Service which will result in a database upgrade being performed on load if the plugins aren't already added to the project.
- Running a sub-model analysis requires two Source processes to be running: the main model project either loaded in the Source UI or through the command line runner; and, the sub-model project loaded through the Source Service. The machine used to run the analysis will thus need to be capable of running thew two processes simultaneously, with enough CPU cores and RAM. Multiple sub-model run results may be held in memory for each single main model run requiring the use of system memory. Using run results streamed to file may help reduce system memory use but will increase physical drive use.