Function: | Sets the forecast length or the start of the run |
---|
Module Name: | ForecastLengthEstimator |
---|---|
Where to Use? | In a workflow |
Why to Use? | To set the length of a forecast or the start of the forecast based on (external) timeseries. |
Description: | The forecastLengthEstimator is a module that can be used at the start of a workflow the set the length of the operations in the other modules in that workflow. |
Preconditions: | the endOverrulable attribute in the relative view period in the time series sets must be set to true in all modules you want to apply the forecast length to |
Outcome(s): | |
Scheendump(s): | link to attached screendump(s) for displays only |
Remark(s): | |
Available since: | DelftFEWS200803 |
Contents
Overview
The forecastLengthEstimator is a module that can be used at the start of a workflow to set the length of the operations in the other modules in that workflow. As most models cannot handle gaps in the input data, this option can be useful if you want to run a hydrological model only with the data available and thus avoid e.g. extrapolating the meteorological forecast data.
Forecast time series
A configuration example of the forecast length estimator using an external forecast time series is given below:
<?xml version="1.0" encoding="UTF-8"?> <forecastLengthEstimator xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews https://fewsdocs.deltares.nl/schemas/version1.0/forecastLengthEstimator.xsd"> <externalForecastTimeSeries> <moduleInstanceId>ImportCOSMO2</moduleInstanceId> <valueType>grid</valueType> <parameterId>P.fg</parameterId> <locationId>COSMO2</locationId> <timeSeriesType>external forecasting</timeSeriesType> <timeStep unit="hour" multiplier="1"/> <relativeViewPeriod unit="hour" start="0" end="30"/> <readWriteMode>read only</readWriteMode> </externalForecastTimeSeries> <minForecastLength multiplier="3" unit="hour"/> <maxForecastLength multiplier="30" unit="hour"/> </forecastLengthEstimator>
The forecast length is defined by the length of the external forecast time series (in this example ImportCOSMO2). You can define a mimimum and / or maximum forecast length (minForcastLength / maxForecastLength). If the actual forecast length of the external forecast looked at is shorter than the minimum forecast length the forecast length is set to this minimum length (in this example 3 hours). If the actual forecast length is longer than the maximum forecast length the forecast length is set to this maximum length (in this example 30 hours).
The logging will provide you with the information which forecast length was used in the run, see example below
[] INFO - Workflow.run - Workflow.ActivityStarted: Workflow 'HBV_FlowForecast_ECMWF'
[] INFO - WorkflowPluginActivity.run - Started Activity ForecastLength_ECMWF
[] INFO - ForecastLengthEstimator.updateForecastLength - ForecastLengthEstimator.Info: Established Forecast Length as 234 hours
Here is a configuration example of the forecast length estimator using a simulated forecast time series. Make sure that your are using the read write mode "read complete forecast" or define a relative view period instead.
<?xml version="1.0" encoding="UTF-8"?> <forecastLengthEstimator xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews https://fewsdocs.deltares.nl/schemas/version1.0/forecastLengthEstimator.xsd"> <simulatedForecastTimeSeries> <moduleInstanceId>HBV_AlpenRhein_COSMO2</moduleInstanceId> <valueType>scalar</valueType> <parameterId>Q.fh</parameterId> <locationId>H-2473</locationId> <timeSeriesType>simulated forecasting</timeSeriesType> <timeStep unit="hour"/> <readWriteMode>read complete forecast</readWriteMode> </simulatedForecastTimeSeries> <minForecastLength multiplier="3" unit="hour"/> <maxForecastLength multiplier="30" unit="hour"/> </forecastLengthEstimator>
Note
The endoverrulable attribute in the relative view period in time series sets must be set to true in all subsequent modules in which you want to use the actual forecast length.
Since 2011.02, the forecastLengthEstimator can also be used to set a fixed end date, e.g. the end of the water year.
<?xml version="1.0" encoding="UTF-8"?> <forecastLengthEstimator xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews https://fewsdocs.deltares.nl/schemas/version1.0/forecastLengthEstimator.xsd"> <timeZone>GMT-12:00</timeZone> <externalForecastTimeSeries> <moduleInstanceId>ImportDATACARD</moduleInstanceId> <valueType>scalar</valueType> <parameterId>MAP</parameterId> <locationSetId>Catchments</locationSetId> <timeSeriesType>external forecasting</timeSeriesType> <timeStep unit="hour" multiplier="6"/> <relativeViewPeriod unit="day" start="0" end="365" endOverrulable="true"/> <readWriteMode>read only</readWriteMode> </externalForecastTimeSeries> <minForecastEndDay>--09-30</minForecastEndDay> </forecastLengthEstimator>
As of 2012.02, the forecast length can be forced to the end of the 2nd wateryear by using a combination of <minForecastLength> and <minForecastEndDay>. The code will first evaluate the minForecastLength before evaluating <minForecastEndDay>.
<minForecastLength multiplier="365" unit="day"/> <minForecastEndDay>--09-30</minForecastEndDay>
requiredExternalForecastTimeSeries
External forecast time series set that must be available in order to run. When not available the run will be skipped. These forecast time series do not affect the T0 of the run.
This can be useful in combination with the time series set element <externalForecastMaxAge> or <relativeForecastSearchPeriod> to make sure there is a forecast available that is not to old.
In the example below a requiredExternalForecastTimeSeries is configured which will prevent the run from executing if there is no forecast available for it that is 1 day or less old.
<requiredExternalForecastTimeSeries> <moduleInstanceId>NotUsedForT0_ButNeedsToBePresent</moduleInstanceId> <valueType>scalar</valueType> <parameterId>par</parameterId> <locationId>loc</locationId> <timeSeriesType>external forecasting</timeSeriesType> <timeStep unit="hour"/> <externalForecastMaxAge unit="day" multiplier="1"/> <readWriteMode>read complete forecast</readWriteMode> </requiredExternalForecastTimeSeries>
Simulated historical timeseries
This module can also be used to determine the start of the period by finding the last (approved) warm state of a specific moduleinstanceId.
<forecastLengthEstimator xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews https://fewsdocs.deltares.nl/schemas/version1.0/forecastLengthEstimator.xsd"> <stateModuleInstanceId>wflow_update_run</stateModuleInstanceId> <stateSelection> <stateSearchPeriod unit="hour" start="-240" end="-1"/> </stateSelection> </forecastLengthEstimator>
Note
The startoverrulable attribute in the relative view period in time series sets must be set to true in all subsequent modules in which you want to use the actual forecast length. Also make sure the statesearchperiod is the same as defined in the ForecastLengthEstimator.
External historical time series
In contrary to external forecast and simulated forcast time series, external historical time series can not be used to determine the forecast length because they are not forecasts. These time series can be configured but will only be used to determine the last time step of measuments and use that time to set the last observation time which in turn can be used to export to a run file or to set the t0 to the latest available time step.
<?xml version="1.0" encoding="UTF-8"?> <forecastLengthEstimator xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews https://fewsdocs.deltares.nl/schemas/version1.0/forecastLengthEstimator.xsd"> <externalHistoricalTimeSeries> <moduleInstanceId>test</moduleInstanceId> <valueType>scalar</valueType> <parameterId>par</parameterId> <locationId>loc</locationId> <timeSeriesType>external historical</timeSeriesType> <timeStep unit="hour"/> <relativeViewPeriod unit="hour" start="-5" end="0"/> <readWriteMode>read only</readWriteMode> </externalHistoricalTimeSeries> <setTime0ToLatestNonMissing>true</setTime0ToLatestNonMissing> </forecastLengthEstimator>
findLatestCommonExternalForecastTime
When your forecast depends on multiple external forecasts (e.g. of an NWP forecast and a Hydro forecast) you may want to harmonize that external forecast time. This can be done by identifying the common external forecast time in the ForecastLengthEstimator and using that as a identifier in the timeseriesset used in the workflow. This identifier determines the external forecast for read and write from and to a time series set.
<?xml version="1.0" encoding="UTF-8"?> <forecastLengthEstimator xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews https://fewsdocs.deltares.nl/schemas/version1.0/forecastLengthEstimator.xsd"> <findLatestCommonExternalForecastTime externalForecastTimeId="test"> <eventCodeOnChange>forecast.newReference</eventCodeOnChange> <dataFeed id="dataFeed1"> <timeSeriesSet> <moduleInstanceId>test1</moduleInstanceId> <valueType>scalar</valueType> <parameterId>par</parameterId> <locationId>loc</locationId> <timeSeriesType>external forecasting</timeSeriesType> <timeStep unit="hour"/> <readWriteMode>read only</readWriteMode> </timeSeriesSet> </dataFeed> <dataFeed id="dataFeed2"> <timeSeriesSet> <moduleInstanceId>test2</moduleInstanceId> <valueType>scalar</valueType> <parameterId>par</parameterId> <locationId>loc</locationId> <timeSeriesType>external forecasting</timeSeriesType> <timeStep unit="hour"/> <readWriteMode>read only</readWriteMode> </timeSeriesSet> </dataFeed> </findLatestCommonExternalForecastTime> </forecastLengthEstimator>
<timeSeriesSet> <moduleInstanceId>ImportTMSI</moduleInstanceId> <valueType>scalar</valueType> <parameterId>P</parameterId> <locationId>Rain93</locationId> <timeSeriesType>external forecasting</timeSeriesType> <timeStep unit="minute" multiplier="10"/> <externalForecastTimeId>test</externalForecastTimeId> <readWriteMode>read complete forecast</readWriteMode> </timeSeriesSet>
<general> <exportType>PI 1.5</exportType> <folder>../junit_test_output/nl/wldelft/fews/system/plugin/dataExport/TimeSeriesExportTest/exportPiExternalForecastTimeId/export</folder> <exportFileName> <name>rainfallPiExternalForecastTimeId.xml</name> </exportFileName> <idMapId>Telemetry</idMapId> <unitConversionsId>UnitConversions</unitConversionsId> <flagConversionsId>FlagConversions</flagConversionsId> <exportMissingValueString>-999.0</exportMissingValueString> <externalForecastTimeId>test</externalForecastTimeId> </general>
time0CardinalTimeStep
Time step which the calculated time 0 should match. When this does not match the run is skipped with status set to redundant.
skipRunWhenTime0EqualToLastRun
When this option is true, after the T0 is determined, it is compared to the T0 of the previous run and if it is equal the run is skipped with status set to redundant.
This can be useful when the T0 is determined by the oldest of multiple different forecast time series which are all created by other workflows which trigger the execution of this workflow.
This way the real execution of the workflow will only happen when all forecasts are new compared to the previous run.
skipRunWhenTimeSeriesMissing
When this option is true, the run will be skipped with status set to redundant, when all the input time series are completely missing.
Sample input and output
Sample input and output
Error and warning messages
Description of errors and warnings that may be generated
Error: | Error message |
---|---|
Action: | Action to fix |
Known issues
Before the 2023.01 a config error will be logged when used inside a parallel block in the workflow.
Related modules and documentation
Links to related parts of the system