Just don’t do it session #2 – Unit test mistakes

Having worked on a few large projects I have learnt to appreciate disciplined unit testing more and more. As such there are a few things that I have seen that mess with maintaining this discipline.

1. Misleading unit test names

I have found a lot of unit tests that end up rotting because they communicate falsely what they do. When maintaining and cleaning up unit tests especially during a bug fix it can be quite frustrating to have to go through a unit test line by line to understand what it does. Hand in hand with this it is important that each test does something very specific avoiding very general unit test names that are not very clear.

2. Allowing failing tests to go unattended

Quite simply, never do this EVER! Much like a large residential building allowing one broken window to go unaddressed will make it easy for continued abuse to creep one until a point is reached when addressing these problem is difficult. I have seen this so many times were tests that were ignored for a month require months of extra effort to get them green again.

3. Creating tests that are affected by the environment they are run in and running sequence

A tricky scenario to pinpoint and address is when you have tests that fail intermittently and developers have to argue often that tests run well on their machine. Lets ignore things that are out of our control such as differences in .NET frameworks installed. There are a number of things that can be neglected that lead to this. A common one is failure to deploy files appropriately and this I have addressed in a recent post. You end up having tests that fail because one runner cannot find the files while others can or tests that share a file and expect it to be clean but one test updates it before the other. Similarly to this you can end up having test that pass/fail based on the run order which really is not in your control. You might wonder how, a common one I have seen is a required dependency that in some cases is there because a certain test ran first and had that resolved and fails when that particular test does not run first. Make it a priority that you tests are isolated and run in all the environments they are required to.

4. Viewing unit tests as tool to find bugs or regression issues

There are many more effective practices such as manual testing and automated integration testing that can address this. Unit tests are more valuable as a development process that verifies the expected behavior of components/unit.

5. Testing performance in unit tests

6. Creating long running unit tests

7. Writing unit tests when the code being tested has become large and complex

8. Expending energy on testing the language/frameworks/trivial code/boiler plate code

9. Testing configuration settings

10. Testing Garbage Collection!

11. Not mocking out externals

 

A few good things to try and do

• Make use of the red-green-refactor approach
• Make use of the arrange-act-assert approach
• Write code that is easy to unit test
• Do code reviews on unit tests
• Make a use of test categories
• Test as part of continuous integration/build process
• Create new tests for every defect
• Run code coverage metrics
• Make use of descriptive messages in asserts
• Write tests that cleanup after themselves
• Ensure that the entire team is clued up on proper unit testing. Consider having a small bootcamp to address this. You entire suite of tests is as good as the worst test in there.

 

Just don’t do it session #1 – Managed resources in destructors!

Don’t cleanup managed resources in destructors!

X DO NOT access any finalizable objects in the finalizer code path, because there is significant risk that they will have already been finalized

When garbage collection kicks in and finalizers are being called there is no guarantee as to what order they will be called in. That means if you have ObjectA composed of ObjectB you cannot know which will have its destructor called first. Meaning if you attempt to clean up ObjectB in ObjectA’s destructor you will be in for the surprise you just saw above, it may be possible that ObjectB was already finalized. There you have it, something that can go terrible wrong when you clean up managed resources in a destructor and it can be very tricky to track this down when it intermittently plays with your mind.

Properly implement the dispose pattern and you will not run into this problem.

Do consider avoiding making any object finalizable in the first place.