Fail Master Page
This page is created to help fulfill the role as Fail Master as efficient as possible. As the role of Fail Master shifts from person to person on daily basis, it is important to share information on findings such that for example Work In Progress tests are not evaluated multiple times by various people.
Build Server Test Categorization
This list exists to aid the current Fail Master in providing an overview of the tests that have been categorized. It is there to provide an overview of what issues are to be solved by the Fail Master, which failing Work In Progres tests are due to being targets for future features or future improvements, and which are due to bugs.
Test: Name of the test. Format is <Test Class>.<Test Method>
Reason/Cause for failure or Work In Progress: Provides an explanation why a test is failing or why the test is set as Work In Progress.
Related Issue: If there is an issue related with the failing test.
Associated Developer: Developer associated with this test. <Developer> is used to indicate that this person has created the test. <Developer> is used to indicate the person is assigned with solving the issue.
Freshness Date: Date when this entry was updated
Related Check-in: Which check-in is related to the test. In italics means it might be associated with this commit. In bold means it is certain this check-in is the cause of the failure.
Test |
Reason/Cause for failure or Work In Progress |
Related Issue (Tools-...) |
Associated Developer |
Freshness Date |
Related Check-in |
|---|---|---|---|---|---|
FunctionFilterTest.FilterUsingMultiComponent |
Missing Feature: Support for filtering multi-component |
- |
- |
24 Sept 2012 |
13596 |
FunctionTest.EventShouldNotBubbleIfReducedFunctionIsUnaffected |
Bug: Event bubbling for filtered functions |
- |
- |
24 Sept 2012 |
15843 |
Missing Feature: Precondition for Arguments not implemented |
Missing Feature: Precondition for Arguments not implemented |
- |
bouvrie |
24 Sept 2012 |
17121 |
FunctionTest.ValuesChangedEventFiredWhileFunctionIsNotConsistent |
Bug/Missing Feature: Incorrect updating of Multi-component Function |
- |
- |
24 Sept 2012 |
17217 |
TableViewCopyPasteTest.PasteIntoNewRow |
Bug: Pasting with Row-Select |
3069 |
- |
24 Sept 2012 |
11269 |
TableViewTest.AddDataToViewUpdatesDatasource |
Bug: Syncing DataBinding does not work properly for TableView |
- |
witlox |
24 Sept 2012 |
12267 |
TableViewTest.ComboBoxTypeEditorWithCustomFormatter_ShouldWork_Tools7594 |
Bug: CustomFormatter on column overrides unspecified TableView CustomFormatter |
7594 |
bouvrie, schre_tn |
24 Sept 2012 |
18999 |
TableViewTest.TableViewCursorKeysWhileInCellEditModeShouldWorkLikeExcel |
Build Server Failure: Hangs on build server |
6647 |
schre_tn |
24 Sept 2012 |
16794 |
HisFunctionStoreTest.HisFunctionStoreWithNetworkCoverage |
Unstable Test: System.IO.IOException : The process cannot access the file 'D:\BuildAgent\DeltaShell\test-data\Plugins\DelftModels\DeltaShell.Plugins.ImportExport.Sobek.Tests\HisData\HisAndNetwork\CALCPNT.HIS' because it is being used by another process. |
- |
- |
24 Sept 2012 |
- |
ProjectQueryContextTest.GetAllByFeatureType_DemoNetwork_CanFindSpecificFeatures |
Pending: Test seems OK-ish, maybe implement additional assertions. TODO: Check with hummel if test is ready to be returned from Work In Progress Checked with Stef: he will fix it |
5920 |
hummel |
27 Sept 2012 |
- |
WaterFlowModelApi1D3DTest.ClearPreviousResultFilesIfNetworkIsInvalid |
Unstable Test: Test green for 3 months, put back in Production and then failed. |
- |
- |
24 Sept 2012 |
- |
BridgeTest.PropertyChangedForTabulatedCrossection |
Missing Feature: PropertyChanged event for changes to TabulatedCrossSectionDefinition property of Bridge and Culvert are not implemented. This is due to CrossSectionDataSet.CrossSectionZWDataTable not having this event and it seems it is not trivial to add it. |
- |
- |
2 Okt 2012 |
- |
DynamicPropertyBagTest.DynamicPropertyBagMaintainsDesiredOrdering |
Missing Feature: DynamicPropertyBad is missing support for PropertySorter applied as TypeConverterAttribute to the properties object |
- |
- |
2 Okt 2012 |
- |
NetCdfFunctionStoreIntegrationTest.ClearingACoverageRemovesCache |
Incomplete Test: Test fails on InvalidOperationException, as the NetCdfFunctionStore object is used without providing a filepath. CollectionChanged event handler has precondition to throw an exception on invalid changes when the NetCdfFunctionStore is not opened. |
3296 |
muurman |
2 Okt 2012 |
9094 |
NetCdfFunctionStoreIntegrationTest.UseExistingNetCdfAsTimeDependendRegularGridCoverage |
Incomplete Test: Fails on Assert.Fail() at start of the test. Commenting this out results in a Rhino.Mocks.Exceptions.ExpectationViolationException for an unexpected call. |
- |
muurman |
2 Okt 2012 |
10932 |
NHibernateMemoryFunctionStoreTest.SaveLoadVariableWithFiltersSet |
Incomplete Test: See TOOLS-5729 |
5729, 5680 |
elzinga |
2 Okt 2012 |
14719 |
NHibernateNetCdfFunctionStoreTest.ModifyingNetCdfFunctionStoreAfterSaveShouldCreateChangesFile |
?Bug/Missing Feature?: Changes file is not created. Should it? Or is it a new feature? If so, what it it's JIRA id? TODO: Check with schre_tn what the story is for this test. |
? |
schre_tn |
2 Okt 2012 |
17016 |
NHibernateProjectRepositoryTest.OpeningAProjectWithWrongVersionGivesException |
???: TODO: Let someone with experience with NHibernate take a look at this test |
- |
muurman |
2 Okt 2012 |
12178 |
RealTimeControlModelTest.ExecutePidRuleFromSobekExample |
Incomplete Test: Test expectencies do not match any more. TODO: Find someone to check expectancies for given scenario |
- |
berg_p |
2 Okt 2012 |
- |
RealTimeControlModelTest.ExecuteRelativeTimeRuleAbsoluteFromSobekExample |
Bug in Test?: Unsure about the test expectencies. There seems to be constant error of 0.5 on the expected values. TODO: Find someone to check expectancies for given scenario |
- |
berg_p |
2 Okt 2012 |
- |
WaterFlowModel1DFiniteVolumeGridTest.FiniteVolumeTest4FiniteVolumeDischargeOutputOnGridPointsWithTimeDependentLateral |
Bug: Expectencies on test are not met, due to mismatch in timestep for lateral discharge output. |
- |
kleijn |
2 Okt 2012 |
15774 |
WaterFlowModel1DPluginGuiTest.WaterFlowModel1DPluginGuiShowADialogWhenMergeActionIsStartedForBranchesOnWhichDataIsDefined |
Bug In Test / Incomplete Test: Bug in the test: mocking a raised event using the incorrect number of arguments. Block of Comments stating that this test needs to be revised. Thirdly, a comment states this test (at the time) failed only on the buildserver |
- |
muurman |
2 Okt 2012 |
16919 |
WaterFlowModel1DSaltTest.TestTimeLagParameter |
Bug or Bug in Test: koste_a commented in revision 18274: |
- |
koste_a |
2 Okt 2012 |
18298 |
WaterFlowModel1DWithLateralSourcesTest.RunInProcessWithLateralSourceTwice |
Bug In test: cal of SetValues() method on a function does not pass precondition in MemoryFunctionStore.SetDependendVariableValues<T>() that blocks assignments like f[1] = new[]{1,2,3}
where the number of components is less than the number of values assigned to the function. |
- |
muurman |
2 Okt 2012 |
16919 |
WaterFlowModel1DModelValidatorTest.BranchesWithTheSameOrderNumberWithOnlyZWCSShouldBeValid |
Incomplete Test: This test was commented out before revision 17297 and was effectively empty. Code was uncommented in revision 17297. TODO: Check with koste_a if this test should be kept, as it is related to branch order number. If so, complete it. |
- |
koste_a |
2 Okt 2012 |
- |
Test back into production
Some (old) tests have been put in Work In Progress and have been resting here for some time. Some of these tests have proven to succeed consistently for some time and are potentially ready to be returned in the regular test-environments. These returned tests do need to be monitored closely by the Fail Master in order to be sure that these tests do no turn into "occasionally failing tests", due to various causes such as state-dependency. Any test that has succeeded consistently for 2 weeks in the does not have to be actively monitored any more and can be removed from the following list.
Test |
Reason back into production |
Check-in |
Date back into production |
|---|---|---|---|
SobekNetworkImporterTest.RunZwolleModelWithAddedCrossSection |
Licence problems do not appear to occur any more. |
19013 |
24 Sept 2012 |
SobekRoughnessImporterTest.ImportReverseRoughness |
Probably fixed in revision 13914 and forgot to remove WIP attribute. |
19013 |
24 Sept 2012 |
PiTimeSeriesLateralSourceImporterTest.ImportSelectedTimeSeriesToLateralSource |
Checked code with janse_ma and test is complete. Test never failed. |
19013 |
24 Sept 2012 |
PiTimeSeriesTargetItemImporterTest. ImportSelectedTimeSeriesToBoundaryCondition |
Checked code with janse_ma and test is complete. Test never failed. |
19013 |
24 Sept 2012 |
WaterFlowModel1DGuiIntegrationTest.DeletingAModelWithNetCDFOutputAfterCreatingANewProjectShouldWork |
Checked code with donchyt and test is complete. Forgot to remove Work In Progress Attribute |
19013 |
24 Sept 2012 |
Open Test Issues
There are a few open issues left to either solve, investigate or monitor. These are listed as follows:
Open ToDo's |
Action |
Freshness Date |
|---|---|---|
WaterQualityModelFileFactoryTest.WriteLoadsFileForLargeNetwork timing upper limit might need to get increased to about 500, in order to keep the mean at around 75% of this limit. |
Monitor (Take action if deemed needed) |
24 Sept 2012 |
The following exception has been popping up (keep in mind that the tmp folder is randomly generated!) :
|
Investigate |
24 Sept 2012 |
ErosionModelTest.ErosionModelPerformance timing upper limit might need to be increased |
Monitor (Take action if deemed needed) |
2 Okt 2012 |
Knipperlicht Test Issues
Details on knipperlicht tests (tests that fail occassionaly). It as useful to keep a list of these tests for quick reference. If worked on (but not resolved) and there is some info that might help in future attempts to solve, please document it here.
Test |
Details |
Freshness Date |
|---|---|---|
WaterFlowModel1DGuiIntegrationTest.ImportInitialConditionWithOpenSideView |
Tried to fix for branch but this is probably not necessary. Test does not come to an end when file 'C:\build.server' is not present. When this file exists the test can be run successfully multiple times. Tried to run on agent 'D00909' under VS2010. Had to copy resharper license there to be able to run unit test. Also had to create 'C:\build.server' file to be able to run test. Also on this machine the test succeeded multiple times. |
27 Sept 2012 |
NHibernateWaterFlowModel1DTest.ImportModelShouldNotTriggerClearingOfOutput |
These tests fail occasionally. I'ved added some extra logging to determine the cause and it shows an access violation exception in ModelInitialize in the CF_DLL. Tried to reproduce by running assembly locally, but to no avail. Not all tests fail always. Test order not conclusive yet about which test is the culprit. |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.Issue1252LoadModelWithNetworkAndWaterLevelShouldNotFail |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.Issue3688NotSavedNetworkAfterBranchIsDeletedAndNetworkHasExecutedShouldNotFailOnReOpen |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.LoadModelAfterRun |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.RetrievingComputationalGridAfterLoadDoesNotClearOutput |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.RunModelAfterLoad |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.RunTwoModels |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.SaveLoadWFM1DWithParameterSettings |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.SavingAModelShouldNotClearData |
"" |
1-okt-2012 |
|
|
|
(extra line below table to allow for easy additions in edit mode)
