Versions Compared

Old Version 10

changes.mady.by.user rachelb (Deactivated)

Saved on

compared with

New Version Current

changes.mady.by.user Teamcity

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Edit through the REST API

Overview

The Replicate Runner runs a single eWater Source model multiple times using an ensemble of time-series inputs . The runner can be used, for example, to run an eWater Source model with a range of alternative climate data inputs. 

There are two means of specifying the time series ensembles:

  1. Generating variants of an existing time series using the Time Series Cycle Creator (TSCC)
  2. Loading a set of pre-generated time series
WarningOption 2 is not currently available for Source Cloud

generated using the Time Series Cycle Creator (TSCC).

eWater Source Project Setup

To run a Replicate job, the eWater Source project must already be created with the time-series to be cycled loaded in Data Sources. It is possible to cycle a single Data Source or multiple Data Sources.

In order to be cycled, the time-series Data Sources must be configured as follows:

  1. Reload on Run must be enabled (which instructs eWater Source to reload the data from a file on disk before simulating the model).
  2. The location of the data file must be specified as a Relative Path, and it must be in the same directory as the eWater Source project file, or a sub-directory of that directory.
  3. The data file must be in the csv file format.
  4. The data files must only contain data covering whole years otherwise cycled data will not match the correct day or month.

For time series that are not to be cycled, Reload on Run must be disabled and the time series files must not be included in the uploaded zip.

Once configured, the eWater Source project and the csv files to be cycled must be placed in a zip archive file.

Info

Note that the project file must be in the root of the zip archive. It must not be within a folder. Do not included any files in the zip other than a single project file and the input csv files to be cycled. In particular do not include other csv files which are not timeseries files as the Time Series Cycle Creator tool will fail to cycle the data and the Job will fail.

Example Replicate Project zip file

ReplicateExample.zip

In this example there are two input timeseries files, Rainfall.csv and Demand.csv, with the latter file being in the Demand directory.

  • ReplicateExample.zip
    • ReplicateTestProject.rsproj
    • Rainfall.csv
    • Demand
      • Demand.csv


Info

The Run Manager Cloud Application is currently configured to use multiple versions of Source and has two plugins available:

  • UrbanDeveloper
  • SubSource

Before running a project on Run Manager it is always recommended to save the Source project using a supported version to avoid needing to upgrade the project during a Source run.

Contents

Table of Contents
maxLevel3
excludeContents
printablefalse

Job Setup

Choose Project Zip

Click "Choose project zip" and select the .zip file that contains your Replicate project. Depending on the size of the file it may take a moment for the file to upload.

Once the project has finished being uploaded, the Replicate Runner configuration options will be displayed.

Source Project

Table 1 lists the eWater Source project configuration options

Table 1. eWater Source project configuration options.
Configuration OptionDescription
ScenarioChoose the eWater Source scenario to be run
Running Configuration

Choose which Source Running Configuration should be simulated.

Currently, the Run Manager is only designed to be used with Single Simulation.

Anchor
TSCC
TSCC
Time Series Cycle Creator (TSCC)

The TSCC generates variants (called replicates) of an existing time series by repetitively 'cycling' the start of the time series to the end. The user can choose whether the data is cycled on a daily, weekly, monthly or annual basis. Table 1 Table 2 lists the TSCC parameters.

Table
1
2. TSCC parameters.
Parameters
Requirement
Description
Number of Replicates
Required
The number of replicates (output time-series) that will be created
Start Cycle Date
Optional

The first data date that will be used by the TSCC to generate the replicates

(data prior to this date is cycled to the end of the time series before generating any replicates)

.

This will default to the start date of the

input time series, if not specified

running configuration.

Start Reporting Year
Replicates
Optional

The year that the replicates will be reported as starting in

.

(

The

the replicates will always use the same start day and month as the input time-series

.)

). The Start Reporting Year cannot be after the Source simulation start date, as this will cause the simulation to fail.

This will default to the

first

year of the

input time-series, if not specified.Cycle Time StepOptional

The interval of data that will be cycled to the end of the time-series to create the replicates. The user can choose between a daily, weekly, monthly or annual interval.

Defaults to annual.

Examples

running configuration's start date.

End Reporting Year

The year that the replicates will be reported as ending in. The End Reporting Year cannot be before the Source simulation end date, as this will cause the simulation to fail.

This will default to the year of the running configuration's end date.


Example

Consider a daily time-series that spans the following 10 year period: 1/01/2000, 2/01/2000, 3/01/2000, ..., 31/12/2009. The data is used by a model that simulates a 4 year period from 01/07/2004 to 30/06/2008.

Running the TSCC with the following parameters would produce 3 4 replicates:

  • Number of Replicates = 34
  • Start Cycle Date = 101/01/20022005
  • Start Reporting Year Replicates = 2010Time Step = Annual2004
  • End Reporting Year = 2008

The start date for all the replicates would be reported as 1 replicate timeseries files will be 01/01/2010 (2004 (the first day of Start Reporting Year) and the end date would will be 31/12/20192008 (the last day of End Reporting Year).

The values of the replicates would be will be taken from the input timeseries files corresponding to the original values as follows:

Replicate 0:1 - the time series values would correspond to the following years of the input time-series: 2002, 2003, 2004, original timeseries files:

2005, 2006, 2007, 2008, 2009

, 2000, 2001

The model would use the original 2005 to 2009 data for the simulation (and would report this as being 2004 to 2008).

Replicate 1:2 - the first year of replicate 0 1 (2000) would be moved to the end. Hence, the values of replicate 1 2 would correspond to the following years of the input time-series: 2003, 2004, 2005, original timeseries files:

2006, 2007, 2008, 2009, 2000

, 2001, 2002

The model would use the original 2006 to 2009 data, plus 2000 data (and would report this as being 2004 to 2008).

Replicate 2:3 - the first year of replicate 1 2 (2001) would be moved to the end. Hence, the values of replicate 2 3 would correspond to the following years of the input time-series: 2004, 2005, 2006, original timeseries files:

2007, 2008, 2009, 2000, 2001

, 2002, 2003.

If a Monthly Time Step was used, then the values of the replicates would be as follows:

Replicate 0:  the values would correspond to the following years of the input time-series: 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2000, 2001.

Replicate 1: the first month of replicate 0

The model would use the original 2007 to 2009 data, plus the 2000 and 2001 data (and would report this as being 2004 to 2008).

Replicate 4 - the first year of replicate 3 (2002) would be moved to the end. Hence, the values of replicate 1 4 would correspond to the following months years of the input time-series: Feb 2002, Mar 2002, ..., Dec 2009, Jan 2000, ..., Dec 2001, Jan 2002.

Replicate 2: the first month of replicate 1 would be moved to the end. Hence, the values of replicate 2 would correspond to the following months of the input time-series: Mar 2002, ..., Dec 2009, Jan 2000, ..., Dec 2001, Jan 2002, Feb 2002.

eWater Source Project Setup

To run a Replicate job, the eWater Source project must already be created with the time-series to be cycled loaded in Data Sources. It is possible to cycle a single Data Source, or multiple Data Sources.

In order to be cycled, the time-series Data Sources must be configured as follows:

  • Reload on Run must enabled (which instructs eWater Source to reload the data from a file on disk before simulating the model).
  • The location of the data file must be specified as a Relative Path, and it must be in the same directory, or a sub-directory, of the eWater Source project file.
  • The data file must be in the CSV file format

For time series that are not to be cycled, Reload on Run must be disabled.

Once configured, the eWater Source project and CSV files (containing the time series to be replicated) must be placed in a ZIP file. The CSV files can be located in sub-directories of the eWater Source project file.

Source Cloud Job Creation

Upload Data

Click "Choose project zip" and select the .zip file that contains your Replicate project. It may take a few seconds for the file to upload.

Image Removed

Configure Job

Table 2 lists the configuration options for the Replicate Runner.

Table 2. Replicate Runner job configuration.
Configuration OptionDescriptionGeneral OptionsScenarioChoose the eWater Source scenario to be runRunning ConfigurationChoose which Running Configuration should be simulatedJob NameSpecify a custom name for the jobTime Series Cycle Creator Options*Number of ReplicatesThe number of replicates (output time-series) that will be createdStart Cycle Date

The first data date that will be used by the TSCC to generate the replicates (data prior to this date is cycled to the end of the time series before generating any replicates).

This will default to the start date of the input time series, if not specified.

Start Year Replicates

The year that the replicates will be reported as starting in. (The replicates will always use the same start day and month as the input time-series.)

This will default to the first year of the input time-series, if not specified.

Cycle Time Step

The interval of data that will be cycled to the end of the time-series to create the replicates. The user can choose between a daily, weekly, monthly or annual interval.

Defaults to annual.

*Refer to the TSCC description for details.

Image Removed

Queue Job

Finally, select Queue to add the Replicate Runner to the queue of jobs to be run.

original timeseries files:

2008, 2009, 2000, 2001, 2002

The model would use the original 2008 and 2009 data, plus the 2000 to 2002 data (and would report this as being 2004 to 2008).

Info

Care must be taken when including timeseries files to be cycled:

  • All timeseries files must cover the same dates otherwise cycling will result in different years being used for different files
  • Only full years of data can be cycled otherwise the cycled data will not match the correct month and day
  • Timeseries files must be in csv format
  • Do not include any additional csv files in the zip which are not used as inputs in the project, particularly non-timeseries files as the TSCC tool will not be able to cycle the data resulting in the job failing


Job Outputs

Choose which outputs the user would like to store and be able to access on completion of the job.

Queue

The Queue enables the user to configure the job details, estimate costs, and send the job to the queue. Table 3 lists the queue options.

Table 3. Queue options.

Configuration Option

Description

Job NameThe name given to the job when viewed in the Run Manager
Agent EndpointsChoose which type of agents should be used to run the job (will be hidden if Run Manager is configured to only have one type of agent endpoint)
Source VersionChoose the Source version to use for the Job
Number of AgentsChoose the maximum number of agents used to run the job
Model Run Estimate (Minutes)(Optional) The user's estimate of how long it will take to run the Source project once
Estimate(Optional) Estimates the costs of running the job based on the user's Model Run Estimate, the selected agents, and the job type.
QueueAdds the job to the queue of jobs to be run


Results

The Replicate Runner will provide the following result files in a zip:

  •  job.log - Log file containing information and messages generated during the Job
  • A folder for each replicate run containing:
    • results.res.csv - The Source results file for all recorders enabled in the Source project
    • Replicate input timeseries files used for the run in the same folder structure as provided in the original zip file