Some points to pay attention during RTC-Tools intergration into FEWS.

Forecast date

For RTC-Tools, the startDate is the earliest timestamp present in a timeseries. This is possibly historical data. The forecastDate is the point from which the optimization/simulation starts. In most cases the startDate and the forecastDate are equal. However, we have to leave the flexibility for the existence of historical information; say, for example, that the decision that you take now depends on what happened in the last 4 hours. In Delft-FEWS the users have a different concept for forecast date; the forecast date of a historical simulation typically is the end time of the simulation. In order to be able to use RTC-Tools for historical data too (i.e. data before the forecast date), without confusing the Delft-FEWS user, one can adapt usage of the forecastDate in the Python script of the model by overwriting the forecastDate with the startDate (therefore, bypassing the forecast date information from Delft-FEWS). That can be done as follows in the pre() method:


def pre(self):
    super().pre()
    self.io.reference_datetime = self.timeseries_import.start_datetime
    ...


Pickle

RTC-Tools is creating a pickle at the first run to store the data about the model. (You can find it in the  model folder). During the following runs the pickle is first compared to the model, and if the model has not changed the pickle is used. It makes the runs faster. However, if the pickle is created from another machine,  it is not compatible with the current version. For this reason it is better not to zip it in the model folder. In order to keep it it can be created an exception when purging.


Ensembles

RTC-Tools does not run if the ensemble id is in the timeseries_import.xml file. Therefore "<ensembleMemberFormat>hide</ensembleMemberFormat>" should be used in the general adapter.


Simulation mode side notes

  • Simulation mode checks at every step if there is a timeserie for a specific variable. If there is a valid value, it assignes that value. This occurs even for timeseries that we would like to modify during the simulation internally, e.g. Q turbine.  Therefore if you want to set a value for a variable in RTC-Tools, make sure that the corresponding timeserie at that timestep is empty (NaN) or does not exist.
  • RTC-Tools simulation sets values for timeseries always to the following time step.
  • There is no built-in check for NaNs! Therefore if there is a NaN value in the input that is needed by the model, the model will go through without saying anything (and uses the value from the previous timestep). This is often unexpected behaviour. If you want to make sure that your model fails in case of absence of necessary data, you have to make this explicit in the Python script.
  • Simulation model is by definition not sending error message even if it fails. Therefore it is possible that a workflow is executed without errors, but the result is an empy timeseries. These issues should be caught by the modeller and add some meaningful error message in the Python code of the RTC-Tools simulation model.
  • No labels

Disclaimer