Fail Master Page
This page is created to help fulfill the role as Fail Master as efficient as possible. As the role of Fail Master shifts from person to person on daily basis, it is important to share information on findings such that for example Work In Progress tests are not evaluated multiple times by various people.
Build Server Test Categorization
This list exists to aid the current Fail Master in providing an overview of the tests that have been categorized. It is there to provide an overview of what issues are to be solved by the Fail Master, which failing Work In Progres tests are due to being targets for future features or future improvements, and which are due to bugs.
Test: Name of the test. Format is <Test Class>.<Test Method>
Reason/Cause for failure or Work In Progress: Provides an explanation why a test is failing or why the test is set as Work In Progress.
Related Issue: If there is an issue related with the failing test.
Associated Developer: Developer associated with this test. <Developer> is used to indicate that this person has created the test. <Developer> is used to indicate the person is assigned with solving the issue.
Freshness Date: Date when this entry was updated
Related Check-in: Which check-in is related to the test. In italics means it might be associated with this commit. In bold means it is certain this check-in is the cause of the failure.
Test |
Reason/Cause for failure or Work In Progress |
Related Issue (Tools-...) |
Associated Developer |
Freshness Date |
Related Check-in |
|---|---|---|---|---|---|
FunctionFilterTest.FilterUsingMultiComponent |
Missing Feature: Support for filtering multi-component |
- |
- |
24 Sept 2012 |
13596 |
FunctionTest.EventShouldNotBubbleIfReducedFunctionIsUnaffected |
Bug: Event bubbling for filtered functions |
- |
- |
24 Sept 2012 |
15843 |
Missing Feature: Precondition for Arguments not implemented |
Missing Feature: Precondition for Arguments not implemented |
- |
bouvrie |
24 Sept 2012 |
17121 |
FunctionTest.ValuesChangedEventFiredWhileFunctionIsNotConsistent |
Bug/Missing Feature: Incorrect updating of Multi-component Function |
- |
- |
24 Sept 2012 |
17217 |
TableViewCopyPasteTest.PasteIntoNewRow |
Bug: Pasting with Row-Select |
3069 |
- |
24 Sept 2012 |
11269 |
TableViewTest.AddDataToViewUpdatesDatasource |
Bug: Syncing DataBinding does not work properly for TableView |
- |
witlox |
24 Sept 2012 |
12267 |
TableViewTest.ComboBoxTypeEditorWithCustomFormatter_ShouldWork_Tools7594 |
Bug: CustomFormatter on column overrides unspecified TableView CustomFormatter |
7594 |
bouvrie, schre_tn |
24 Sept 2012 |
18999 |
TableViewTest.TableViewCursorKeysWhileInCellEditModeShouldWorkLikeExcel |
Build Server Failure: Hangs on build server |
6647 |
schre_tn |
24 Sept 2012 |
16794 |
HisFunctionStoreTest.HisFunctionStoreWithNetworkCoverage |
Unstable Test: System.IO.IOException : The process cannot access the file 'D:\BuildAgent\DeltaShell\test-data\Plugins\DelftModels\DeltaShell.Plugins.ImportExport.Sobek.Tests\HisData\HisAndNetwork\CALCPNT.HIS' because it is being used by another process. |
- |
- |
24 Sept 2012 |
- |
ProjectQueryContextTest.GetAllByFeatureType_DemoNetwork_CanFindSpecificFeatures |
Pending: Test seems OK-ish, maybe implement additional assertions. TODO: Check with hummel if test is ready to be returned from Work In Progress Checked with Stef: he will fix it |
5920 |
hummel |
27 Sept 2012 |
- |
WaterFlowModelApi1D3DTest.ClearPreviousResultFilesIfNetworkIsInvalid |
Unstable Test: Test green for 3 months, put back in Production and then failed. |
- |
- |
24 Sept 2012 |
- |
BridgeTest.PropertyChangedForTabulatedCrossection |
Missing Feature: PropertyChanged event for changes to TabulatedCrossSectionDefinition property of Bridge and Culvert are not implemented. This is due to CrossSectionDataSet.CrossSectionZWDataTable not having this event and it seems it is not trivial to add it. |
- |
- |
2 Okt 2012 |
- |
Test back into production
Some (old) tests have been put in Work In Progress and have been resting here for some time. Some of these tests have proven to succeed consistently for some time and are potentially ready to be returned in the regular test-environments. These returned tests do need to be monitored closely by the Fail Master in order to be sure that these tests do no turn into "occasionally failing tests", due to various causes such as state-dependency. Any test that has succeeded consistently for 2 weeks in the does not have to be actively monitored any more and can be removed from the following list.
Test |
Reason back into production |
Check-in |
Date back into production |
|---|---|---|---|
SobekNetworkImporterTest.RunZwolleModelWithAddedCrossSection |
Licence problems do not appear to occur any more. |
19013 |
24 Sept 2012 |
SobekRoughnessImporterTest.ImportReverseRoughness |
Probably fixed in revision 13914 and forgot to remove WIP attribute. |
19013 |
24 Sept 2012 |
PiTimeSeriesLateralSourceImporterTest.ImportSelectedTimeSeriesToLateralSource |
Checked code with janse_ma and test is complete. Test never failed. |
19013 |
24 Sept 2012 |
PiTimeSeriesTargetItemImporterTest. ImportSelectedTimeSeriesToBoundaryCondition |
Checked code with janse_ma and test is complete. Test never failed. |
19013 |
24 Sept 2012 |
WaterFlowModel1DGuiIntegrationTest.DeletingAModelWithNetCDFOutputAfterCreatingANewProjectShouldWork |
Checked code with donchyt and test is complete. Forgot to remove Work In Progress Attribute |
19013 |
24 Sept 2012 |
Open Test Issues
There are a few open issues left to either solve, investigate or monitor. These are listed as follows:
Open ToDo's |
Action |
Freshness Date |
|---|---|---|
WaterQualityModelFileFactoryTest.WriteLoadsFileForLargeNetwork timing upper limit might need to get increased to about 500, in order to keep the mean at around 75% of this limit. |
Monitor (Take action if deemed needed) |
24 Sept 2012 |
The following exception has been popping up (keep in mind that the tmp folder is randomly generated!) :
|
Investigate |
24 Sept 2012 |
Knipperlicht Test Issues
Details on knipperlicht tests (tests that fail occassionaly). It as useful to keep a list of these tests for quick reference. If worked on (but not resolved) and there is some info that might help in future attempts to solve, please document it here.
Test |
Details |
Freshness Date |
|---|---|---|
WaterFlowModel1DGuiIntegrationTest.ImportInitialConditionWithOpenSideView |
Tried to fix for branch but this is probably not necessary. Test does not come to an end when file 'C:\build.server' is not present. When this file exists the test can be run successfully multiple times. Tried to run on agent 'D00909' under VS2010. Had to copy resharper license there to be able to run unit test. Also had to create 'C:\build.server' file to be able to run test. Also on this machine the test succeeded multiple times. |
27 Sept 2012 |
NHibernateWaterFlowModel1DTest.ImportModelShouldNotTriggerClearingOfOutput |
These tests fail occasionally. I'ved added some extra logging to determine the cause and it shows an access violation exception in ModelInitialize in the CF_DLL. Tried to reproduce by running assembly locally, but to no avail. Not all tests fail always. Test order not conclusive yet about which test is the culprit. |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.Issue1252LoadModelWithNetworkAndWaterLevelShouldNotFail |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.Issue3688NotSavedNetworkAfterBranchIsDeletedAndNetworkHasExecutedShouldNotFailOnReOpen |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.LoadModelAfterRun |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.RetrievingComputationalGridAfterLoadDoesNotClearOutput |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.RunModelAfterLoad |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.RunTwoModels |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.SaveLoadWFM1DWithParameterSettings |
"" |
1-okt-2012 |
NHibernateWaterFlowModel1DTest.SavingAModelShouldNotClearData |
"" |
1-okt-2012 |
|
|
|
(extra line below table to allow for easy additions in edit mode)
