Data Import and Export

Owned by rachelb (Deactivated)
Last updated: 16 Dec 2020 by Juanita Moolman
4 min read
Note: For simplicity, any loaded time series (*.res.csv or other formats) are called results.

Loading results

Results can be loaded into the Results Manager using:

There are several ways to load data:

  1. Load Results Files
  2. Load Time Series
  3. Load Source Database

These methods are described below.


Load Results Files

Loads all *.res.csv files in a specified folder (even if there are other time series in the folder). This is non-recursive, ie. it does not load files within sub-folders. The loaded files are listed under Loaded Result Files at the bottom of the tree menu.

Load Time Series


Loads any time series file type compatible with Source (refer to a complete list in Data file formats). Multiple files can be selected and imported at once. The imported files are listed under Loaded Time Series at the bottom of the tree menu. Just like data sources, if a file has multiple columns, all columns will be imported. Unlike Data Sources, there is no choice to only import some of the data columns.

A Source Database (.sdb) can also be loaded as a time series, but cannot be queried to filter the data loaded. To access this option see Load Source Database below.

To set units on loaded time series, select the time series file name. The right side of the Results Manager window will display the Unit Preferences options as shown in Figure 1 below. To select the preferred unit, click on 'no unit selected' and then choose from the list of available units.

Figure 1. Unit Preferences for Loaded Time Series



Load Source Database

Loads a Source Database (SDB) file. The SDB is Source's binary file format for storing time series data and associated metadata (for details see the SDB file format documentation). A Source Database previously exported from Result Manager can be loaded using the 'Load Source Db' option in Results Manager. The newly loaded SDB files will be listed under Source Database Result Files at the bottom of the Results Manager tree menu. An SDB can also be loaded as a time series but this doesn't provide the ability to filter according to metadata keys (see SDB Importer Details). 

SDB Importer Details

Selecting Load Source Database opens the Source Database Importer (Figure 2). Click the three dots "..." in Source Database Location field to navigate to and select an existing *.sdb file from your hard drive.

By default, all time series in the SDB file will be imported. However, the user can specify Search Criteria to import time series with specific metadata content. To filter the data based on the values of a particular Metadata Key:

  1. Select the Apply Filter checkbox
  2. Enter the text to be matched in the Search Text field

A detailed description of the metadata keys listed in Figure 2 can be found in the SDB file format documentation. Only time series matching the search text for the selected metadata key will be imported. A match occurs if all or part of a string is matched, and matches are not case sensitive. For example, the search text "Flow" could match "Downstream Flow" and "Inflow".

If the SDB is filtered on multiple metadata keys simultaneously, then only time series that match all search criteria will be imported.

Figure 2. Editor for importing and querying SDB files.

Exporting results

To export a result, result set or custom chart, right click on it in the tree menu panel and choose one of the export options from the contextual menu:

  • Export Time Series – this allows you to save the result(s) as a *.res.csv file, which can then be re-imported for future use maintaining the same data structure.
  • Export to Excel – opens the result(s) in Excel, allowing you to save a *.csv file.
  • Export to Source Database – exports results to a Source Database (*.sdb) file. Further details are provided below.

For single results, most export methods allow you to export for either the Current Run only or All Runs. All Runs will export that result for all runs in the current session.

Also, for Time Series exports, there is an option to export either the Raw or Transformed results.  The Transformed export will include the missing data value (-9999) for any results filtered by the transform applied, and will also export results as aggregated in the aggregation time step.  Note that this applies only to configured Transforms not transforms applied using the Quick View Toolbar.  Raw time series results export will include all results.

Note: When you right click on any entry in the tree menu, apart from a single result, you can choose to export data for all series beneath it. So to save all results for a scenario, right click on the scenario name. This will export results for a scenario, which is not the same as saving a run in your project.

Export to Source Database

When exporting to a Source Database (*.sdb) file, the user has the option to:

  • Export to New - create a new *.sdb file, or
  • Append to Existing - append to an existing *.sdb file.

When appending to an existing file, duplicate time series are overwritten. Time series are considered to be duplicates if the following metadata fields match exactly:

  • ScenarioName
  • Timestep
  • StartDate
  • EndDate
  • Site
  • Structure
  • Units
Warning: It is advisable to back up you Source Database before appending new data.