I’ve been testing some code I’ve written for sending out notification emails according to a moderately complex set of rules. So, what have I learnt?
- Developers make rubbish testers. We know our own code too well, and already have expectations of how things should work.
- Testing has too many variables for any reasonable size application. You can’t test exhaustively. Equally, testing only what you think needs testing will also miss things. I’m beginning to think some automated, semi random testing (if you can) is the way forward – and you’ll still miss things.
- Specialist testers are good. I’m convinced a specialist would see what I’ve missed, and have good ideas over how to test generally. I don’t think I’m alone in that..
- Automated test is not a panacea… I really don’t see how it could apply in this case, at least not without a unit test program that could create database records, read a SharePoint library, and read an exchange server. All reasonable – but how do I then test the test app? ‘Cos it’s a complicated as my actual application.
- … and Unit testing isn’t a panacea either… I know, they’re related, but this does at least imply that you’re testing, well, small units of code. That relies on problems not occuring because of mismatches in the interoperation between units.
- … and with them both, you have to remember that you’re designing the tests. If you didn’t think of the case to handle it in your code, do you think you’ll think of it to test for it?
All that said, my application seems to work pretty well. Just there was a lot of manual cross referencing of results to check things worked, so I had a lot of time to think.
I’d really like a project where I can get thoroughly stuck into unit testing. Didn’t use it in this one, but I did think about how I would’ve done it – and I suspect I’d have missed one of the bugs I found. I suspect that unit testing’s main advantages are in forcing developers to actually think up front (sometimes rare), and in ensuring a consistent public interface despite internal changes. But I don’t see how you can get away from the fact that someone has to site and decide what to test – and that way things can be missed..