What's your testing strategy?

When do you decide to start testing? What do you start testing for? What are things you are looking for and what are things you don't look for when you test?

Comments

[Ben Halpern]

Testing plays a pretty huge role in everything we do. Strategically we use testing for:

• Testing for expected behavior in the context of merging/deploying new code. Tests give us the confidence to say yes to new things. Not sure what we'd do without a solid amount of tests.
• Measuring quality via code coverage and other metrics. Code coverage is a dubious metric because it can lead to false confidence, but it is a way for us to sniff out ways to improve quality of our code. Code coverage is associated with quality even if there are gotchas.
• Test driven development helps map out problems and ultimately get to certain solutions much quicker. Not sure where to start? Write a test.
• Testing is a lot like documentation as well. It's probably the best way to see how something is supposed to work when docs are lacking or the code is unclear.

I'm no heavy-handed evangelist of any of this stuff, but I'd say our strategy is mostly to achieve the points listed here, with plenty of laziness or hurriedness mixed in along the way.

[Jesse Phillips]

We do a lot of integration with 3rd party and client software. This ends up providing me with an additional level to catch issues in integrated environments. I do take advantage of this, meaning I don't always investigate integration requirements or content as well as I could to catch issues. But what I do end up with lets me track down issue by reproducing them quickly.

I find that clients a much more willing to see issues if the resolution and build is fast. Obviously things can be much different when end-user are the client.