Funny - just today I read that the great Donald Knuth, in a recent interview, dissed unit-testing:
"As to your real question, the idea of immediate compilation and "unit tests" appeals to me only rarely, when I’m feeling my way in a totally unknown environment and need feedback about what works and what doesn’t. Otherwise, lots of time is wasted on activities that I simply never need to perform or even think about. Nothing needs to be "mocked up."
(In this interview he also talked against multi-core paradigm (which may be the only way to keep up with Moor's law due to our CPUs getting hotter and hotter), but that's a different story).
Why is this funny? Because just today at work I installed JetBrains' TeamCity - a very fine unit test and automated build suite. Sadly I didn't have time to finish configuring it today, but I already see the great benefit this will bring to all our development team. It goes beyond a simple automated build system (which is important enough, making sure no member of the team messes something up by accident, opening up new venues for brave code refactoring).
TeamCity aims to take all resource-intensive activities off the developer's computer. Just some of its features are (I didn't have the chance to test them, but I believe JetBrains - these are the guys that make Resharper after all):
- Code Analysis (finding code and style errors)
- Code Duplication - automatically find the copy-pastes I hate so much, with configurable granularity (java only, coming soon for .NET)
- Pre-commit Build - Suppose it is now 8PM, you're about to leave for home, and unfortunately you have a rather large piece of uncommited (checked out) code. What do you do? You do not commit the code. Why? Because there's a good chance it will screw up the build and you don't want to leave the common integration area messy. To the rescue comes the Pre-commit Build feature - you can commit the code, have TeamCity catch your commit before it actually happens, test it automatically on a build agent (dedicated build machine), and then only if the code compiles and all tests pass it will check in the code for you. You do not have to wait for tests anymore! This frees up developer's time for actual development, while preserving the high benefits of unit-testing (that Knuth doesn't see, for some reason).
- I'm sure it has load of other features I haven't dug up yet - among them the ability to run tests without any commit process "just for fun", it's ability to smoothly handle multiple Agents (each running a configurable number of builds concurrently), supporting multiple source controls, IDEs and development languages, ICQ/RSS/Jabber support, and last but not least a button "Run this test on the fastest available test machine".
So far the systems I've used for unit-testing were:
- Manually running unit-tests a couple of times, then forgetting about them :)
- A patched-up system that automatically runs tests for you on checkin, with a build status indicator that Gal Golan in my development crew in the army wrote himself
- CruiseControl.NET - an open source project that does something similar to the above, only slightly more configurable
As you can see by the length of this post, I'm excited to try on a professional solution to this problem for once :) And for those of you that got this far, let me add that it's free or charge - for up to 3 Build Agents and 20 developers. Huzza!