Step 4. Setup ModuleDataSets and Module Adapter

Within this step, one organizes the ModuleDataSets (i.e. the Modflow-files) and develops the instruction-file for the Modflow Module Adapter. More detailed background information is provided
in a seperate page.

Configurations of release 1.4.2 (PR10) need to be upgraded. See Upgrade ModuleDataSet handling.

Assumptions for this step-by-step explanation: Any reference to XXX should be replaced by the model abbreviation as used within the configuration.

Step 4.1. Prepare Modules-directory

1. Unzip the contents of the SampleRegionModules.zip file into the Modules-directory.
2. Rename directory \Modules\Mfaaa into \Modules\Mf96 for Modflow96 or \Modules\MFvkd for ModflowVKd or variants (e.g. MFSSQ02).
3. Make sure you have the latest version of the Modflow Module Adapter Modflow Module Adapter.zip

Step 4.2. Populate the sub-directories

Go to #Step 4.3. Update the NAM file

Within the NGMS, a distinction is made between Modflow files that can be changed by the Module Adapter and Modflow files that are not changed. Currently, the Modflow Module Adapter may changes the following Modflow files:

• BAS, to update the initial heads and or the stress periods (simulation length)
• RCH, to update the stress periods
• WEL, to update the abstraction time series and the stress period
• STR, to update stream flow time series and available stress periods
• DRN, RIV, GHB, to skip stress periods when required

Note: In the near future, the STR file may need to be manipulated by the Modflow Module Adapter as well.

For each scenario type, one needs to repeat the following sub-steps:

1. Copy all Modflow data files from the DVD with approved data sets into the \baseData directory
2. Rename the NAM-file into xxx.org.NAM-file to preserve this file and its internal file references for back tracking purposes
3. Except for the NAM-file, remove the run-number all data files in the \baseData-directory, and ensure that the naming convention has become 'xxx.file extension' where xxx should be replaced by the model abbreviation.
4. Copy the NAM file to the root of the scenarioType-directory and rename to xxx.NAM where xxx should be replaced with the ModelAbbreviation
5. Zip all files in the \baseData directory into a file with name XXX_runMF_ref(yyyy)_baseData.zip where XXX refers to the model abbreviation and YYY refers to the full scenario name (Historic, Naturalized, RecentActual, RecentActualS, FullyLicenced, FullyLicencedS or LongTermAverage)

Note: for release 1.4 and before this step used to be:

1. Copy all Modflow data files except BAS, WEL and RCH from the DVD with approved data sets into the \baseData directory
2. Rename the NAM-file into xxx.org.NAM-file to preserve this file and its internal file references for back tracking purposes
3. Except for the NAM-file, remove the run-number all data files in the \baseData-directory, and ensure that the naming convention has become 'xxx.file extension' where xxx should be replaced by the model abbreviation.
4. Copy the NAM file to the root of the scenarioType-directory and rename to xxx.NAM where xxx should be replaced with the ModelAbbreviation

Step 4.3 Update the NAM file

Go to #Step 4.4 Prepare offset file

Note: please update the NAM file as delivered with the model and do not take the XXX.NAM file as given in the SampleRegion. The NAM-file of the sample region may refer to Modflow files or IUNIT sections which do not correspond to the model being uploaded.

Update the NAM file as follows:

1. the Modflow.OUT file should point to the logs-directory
2. all other input files should point to the workMF directory
3. all output files should point to the workMF directory

For release 1.4 this used to be

1. the Modflow.OUT file should point to the logs-directory
2. the BAS, WEL, RCH files should point to the workMF directory
3. all output files should point to the workMF directory
4. all other files should point to the baseData-directory.
   
   
   

Step 4.4. Prepare offset file (OFS)

Go to Step 4.5 Prepare ModuleDataSet.zip-files

The OFS-file provides the Module Adapter context information on the geographic position of the model and the calendar associated to the run. (More details .)

The following steps need to be conduct

1. Modify the sample WMW.OFS-file such that it fits your model. Please pay specific attention to the LL cell position and the start date of the run.
   Note: At this moment, the XML time step descriptor does not need to be defined, but in the near future this may change.
2. Save the file as xxx.OFS - where xxx refers to the model abbreviation - at any place.
3. Add the OFS file to the baseData.zip file stored in directory \baseData

Step 4.5. Prepare ModuleDataSet-zip files

This step can be skipped from release 1.5 onwards

Go to #Step 4.6. Prepare model states for testing

Each scenario-type - both default and what-if - has its own ModuleInstance to execute Modflow. To ensure that each Modflow run starts with the original data files, a ModuleDataSet-zip file is created which holds the original RCH and WEL file, as well as the OFS file holding the calendar information for this run. ModuleDataSet.zip files are stored in the \Config\ModuleDataSetFiles-folder.

The Naming convention applied to the zip file is 'ModuleInstanceId version default.zip', e.g. WMW_runMF_ref(Historic) 2.00 default.zip

For each scenario type (Historic, Naturalized, RecentActual, FullyLicenced, LongTermAverage), one needs to repeat the following sub-steps:

1. Create a temp-directory for this scenario type (e.g. at the region-root)
2. Copy the Modflow data files BAS, RCH, WEL from the DVD with approved data sets into the temp-directory
3. Copy the OFS-file as created in step 4.4 into the temp-directory
4. Remove the run-number of all data files in the temp-directory, and ensure that the naming convention has become 'xxx.file extension' where xxx should be replaced by the model abbreviation.
5. Ensure that the OFS-contents (start date) matches the approved data set
6. Zip the RCH, WEL and OFS files from the temp-directory into a file with name 'XXX_runMF_ref(ST) v.vv default.zip' where XXX is the model abbreviation, ST is the scenario-type and v.vv is a version number. E.g. TI_runMF_ref(Historic) 1.00 default.zip
   • Please ensure that no spaces are left in the ModuleInstanceId
   • Please ensure the the BAS file is left behind

Repeat the following steps for scenario types (if available) RecentActual, FullyLicenced, LongTermAverage

• Copy zip files while replacing 'ref' by 'whatif'.

Finally:

• Move all ModuleDataSet-zip files from the temp-directory to the \Config\ModuleDataSetFiles-directory.

step 4.6 Prepare model state-zip files for testing purposes

Go to #Step 4.7. Update Module Adapter instructions

Each scenario-type - both default and what-if - has its own model state zip-files holding the Modflow BAS-file as well as a (renamed) copy of the 'original' OFS-file for back tracking purposes. (NB the OFS file stored in the ModuleDataSet-zip file will be manipulated by the Module Adapter).

The Naming convention applied to the state-zip file is 'ModuleInstanceId ColdStateId.zip', e.g. WMW_runMF_ref(Historic) InitialCold.zip.
The state-zip files are stored in the \ColdStates directory.
In the following steps three of the seven ColdStates-files are perpared. the others are prepared in step 4.8

For each scenario type (Historic, Naturalized, RecentActual, FullyLicenced, LongTermAverage), one needs to repeat the following sub-steps, each one in its own temp-directory as created in the previous step:

1. Create zip-file for production purpose: InitalCold
   • Rename the OFS file from 'XXX.OFS' into 'XXX.original.OFS', with XXX being the model abbreviation
   • Zip the BAS and the original.OFS files from the temp-directory into a file with name 'XXX_runMF_ref(ST) InitialCold.zip' where XXX is the model abbreviation, ST is the scenario-type. E.g. WMW_runMF_ref(Historic) InitialCold.zip
2. Create zip-file for testing purpose: Test(short)
   • Open the BAS file and reduce the simulation length to e.g. 2 years (number of stress periods = 4th item on line 3). It is recommended to add a statement in the second line while indicates the original length of the simulation. Save the BAS-file again
   • Zip the BAS and the original.OFS files from the temp-directory into a file with name 'XXX_runMF_ref(ST) Test(short).zip' where XXX is the model abbreviation, ST is the scenario-type. E.g. WMW_runMF_ref(Historic) Test(short).zip
3. Create zip-file for testing purpose: Test(medium)
   • Open the BAS file and adjust the simulation length to e.g. 5 years (number of stress periods = 4th item on line 3). Save the BAS-file again
   • Zip the BAS and the original.OFS files from the temp-directory into a file with name 'XXX_runMF_ref(ST) Test(medium).zip' where XXX is the model abbreviation, ST is the scenario-type. E.g. WMW_runMF_ref(Historic) Test(medium).zip

Finally:

• Move all state-zip files from the temp-directory to the \ColdStates-directory.

NB Pick a simulation length for the tests, such that one will not loose too much time when running workflows for testing purposes. Just a few steps is sufficient for testing and debugging.

{anchor:Step 4.7. Update Module Adapter instructions

step 4.7 Update Module Adapter instructions (references)

Go to #Step 4.8. Obtain configuration data via Module Adapter
Go to Background on the Module Adapter

To update the Module Adapter instructions files, repeat the following sub-steps for each scenario type:

1. For all root-files of the Scenario-type-folder (e.g. the folders \Hist, \Nat, \RecAct, \FulLic, respectively \Lta) replace XXX by the model abbreviation.
2. Update the contents of all root-files by replacing XXX with the model abbreviation
3. Check the parameters listed in the BINARY MAPSTACKS command against the parameters produced by the model
4. Ensure that 'merge' commands are applied for multi-layer models, both for the BINARY MAPSTACKS command and the WELLS_OUT command
5. for the XXX_input.in files, ensure that the SKIP command is added for the DRN, RIV and/or GHB package if relevant. This commands skips stress periods if required

step 4.8 Obtain configuration data via Module Adapter

Go to #Step 4.9. Prepare model states for production

To enable proper mapping between locations as known by the system, and the locations as known by the Module Adapter (e.g. wells and stream discharges), some information needs to be generated by the Module Adapter. A set of steps need to be conducted for each scenario-type.

Repeat the following sub-steps for each scenario type:

1. Unzip the ModuleDataSet-zip file - as created in step 4.4 - into the \workMF folder of the associated scenario-type
2. Unzip the Test(short)-zip file - as created in step 4.5 - into the \workMF folder of the associated scenario-type
3. Run the XXX_prepare.bat file
4. Check the diagnostics-file in the \logs-directory.
   Under normal completion conditions the last line should state: 'No further instructions: Ending job'
5. check the toNGMS-directory. This folder should hold:
   • strdir.asc: the stream directions file holding in PCRaster local drainage network format (derived from STR file)
   • XXX_Stream_Cell_Locations.xml: an XML-file in Locations-format holding the individual Ids and positions of stream cells (derived from STR file)
   • XXX_Stream_Discharge_Locations.xml: an XML-file in Locations-format holding the individual Ids and positions of stream cells having a time series associated with discharge/abstraction/runoff data (derived from STR file)
   • XXX_Stream_Cell_Locations_Grid.xml: an XML-file in irregular Grid-format holding the positions of stream cells (derived from STR file)
   • XXX_River_Cell_Locations_Grid.xml: an XML-file in irregular Grid-format holding the positions of stream cells (derived from RIV file)
   • XXX_Drainage_Cell_Locations_Grid.xml: an XML-file in irregular Grid-format holding the positions of stream cells (derived from DRA file)
   • XXX_Stream_Discharge_Locations_Grid.xml: an XML-file in irregular Grid-format holding the positions of stream cells having a time series associated with discharge/abstraction/runoff data (derived from STR file)
   • XXX_GWabstractions_Locations.xml: an XML-file in Locations-format holding the individual Ids and positions of individual abstractions having a time series in the WEL file.
   • XXX_GWabstractions_Locations_MultiLayer.xml: an XML-file in Locations-format holding the individual Ids and positions of multi-layer abstraction, which will have a time series associated for all abstractions at the same Row-Column position

Typical errors:

• diagnostics file reports no proper completion (i.e. stack dump) due to:
  • one of the Modflow input files as mentioned in the NAM-file is not properly in place
  • the directories mentioned in the IN-file are not in place
    If none of the above two issues causes the problem, one should run the batch-file from a DOS-prompt, or one should extend the call by adding the statement '>logs\prepare.log'. This will put the system out in a file prepare.log, placed in the logs directory

step 4.9 Prepare model states for production purposes

Go to #Step 4.10. Update Module Adapter for output

The following model states (i.e. initail heads) are identified for 'production' runs:

• InitialCold: model state as delivered. The model might need a warm up period
• InitialWarm: model state after warm up period of model (typically 01-01-1970)
• MostRecent: model state at end of default run handed over.

In addition to these states, intermediate heads are recognized to do assessments for different hydrologiolca conditions:

• AvgConditions: heads under average hydrological conditions
• DryConditions: heads under dry hydrological conditions
• WetConditions: heads under wet hydrological conditions

In step 4.5, the zip-file for the InitialCold state has been prepared.

To create the zip-files for the other state conditions, one should do the following steps (for each scenario-type):

1. Ensure that the RCH, WEL, the original BAS file and the OFS file are in the \workMF directory
2. Run the model for full length using the XXX_run.bat file (do not delete HDS file)
3. Update the instructions to the Module Adapter in the XXX_bastemplate.in file as follows:
   • Check the dates on the representative hydrological states, as given in Data-spreadsheet, against dates of a TimeSeries.xml or Mapstacks.xml file, generated in step 4.7.
   • Update BAS_TEMPLATE command of xxx_basttemplate.in with the appropriate dates as listed a Timeseries or MapStacks.xml file
4. Generate BAS files with Module Adapter
5. For each BAS file:
   • rename to XXX.BAS where XXX refers to model abbreviation
   • create XXX.<ColdStateId>.OFS, where the contents of the file is updated to reflect the appropriate date
   • Zip the BAS and the original.OFS files from the temp-directory into a file with name 'XXX_runMF_ref(ST) ColdStateId.zip' where XXX is the model abbreviation, ST is the scenario-type. E.g. WMW_runMF_ref(Historic) AvgConditions.zip
6. Copy zip-file to ColdStates directory

step 4.10 Update Module Adapter instructions for output (parameters)

While the majority of the update is done is step 4.6, this step is to check which output parameters are /should be included in NGMS.

To do this check, execute the following steps (if needed for each scenario-type):

1. open file XXX_output.in and ensure that the BINARY_MAPSTACKS command does not list any parameters (NB the Module Adapter will then produce all parameters as available in the Modflow output files)
2. run Module Adapter (on a directory whch contains Modflow output)
3. investigate the contents of the 'toNGMS' directory
4. decide which parameters to include in NGMS
5. list parameters after BINARY_MAPSTACKS command

Be aware:

• each parameter which is exported by the Module Adapter, but not imported by a GeneralAdapterRun is a waste of time and disk space
• each parameter imported by the GeneralAdapter must also be postprocessed and displayed, else it is useless