Friday, December 16, 2011

Known way of managing ET #01 - Stealthily

tl;dr – some people hide their best work from their paymasters

A few people get together to find problems in snatched moments. There's little or no imposed direction, measuring, or task control, and rarely any sense of completeness or coverage. Although the work sometimes gets done with tacit support from one or two individuals in the upper echelons, there is little oversight and it is usually a hidden activity. Testers don’t log time or bugs through the usual channels, and it feels almost like an indulgence, a guilty pleasure.

I've been on a team who hid a pair of stealthy explorers in a corner for a few hours each week. The exploratory testers would tell the rest of the team about the bugs they’d found. Generally those bugs were 're-found' during manual scripted testing to allow them to be logged within the imposed structures of the project – and if no script might find a particular bug, we would assemble one. The customer would not countenance paying for unscripted testing, but was very impressed that we were designing such effective scripts*.

Exploratory testing becomes stealthy typically because those who control and budget for team members' time don't approve of looking for trouble. Look out for rigid 'verification and validation' contracts, consultancy contracts that only allow a small set of explicitly-approved billable activities and legal fears of explicitly-acknowledged defects.

I've seen Exploratory Testing hidden most commonly in teams that focus on user acceptance and regression testing, but I've also seen it in a self-labelled agile team that relied on a (rather sparse) set of low-level confirmatory automated tests. These teams tend to be a mix of self-identified testers and people who may be seconded into the test team or are otherwise keen to avoid the label. However, I’ve even seen stealthy approaches in test teams who were exploring with a degree of management support, but who felt that some of their approaches were beyond what might be accepted.

However nasty such hidden work looks from the outside, it's often rather well supported by individuals within the testing teams. People get the opportunity to work heroically, to subvert management decisions (especially gratifying if those decisions feel irresponsible), and sometimes to have a direct link to someone rather higher up in the tree of project status. The stealthy effort tends to get a geekily-sexy label.

When I meet experienced exploratory testers who have carved themselves a niche in some monolithic institution, they're often proud to be stealthy, and sometimes unhappy to share their approaches. Sometimes their reticence is justified – in groups which insist they don't do ET, acknowledging that you're an the ET specialist doesn't necessarily improve your day.

It’s worth mentioning that some companies consciously take a stealthy approach to discovery work so that they have plausible deniability, for instance while finding bugs to get a company out of a contract, or finding bugs that no one who could be legally-liable should know about. Such activity will challenge your ethics. Call time on these if you must – and sometimes, you must – but be aware that you may be shouting at the waves.

* This was in the early nineties. Don’t think I would put up with it now – but don’t think that the practice has ended, either.


  1. In your experiences, are there discipline specific reactions to ET? Do developers tend to respond one way and PMs another?


  2. Very, very broadly, PM's don't like open-ended stuff or surprises, but are happier with gambles than some people expect, and do like diligent problem-finding done early enough. People who make stuff often have an innate bias against criticism of their stuff (don't we all), but do like collaboration and feeling that their stuff has been made to be robust.

    ET /can/ do all of this, but I'm not particularly sure what you mean. Can you give me an example?

  3. Stealth exploratory testers - so true. I took this role once when I had to prove to my team and the client just how valuable exploratory testing is. Once the client acknowledged what we were upto they were more than happy to ask me do exploratory testing than executing test cases. To me it's how you share your exploratory session/ story that makes the difference :)

  4. ... and who you share it with. Your primary supporter might not be the budget-holder, or might not be in a position to take the office politics / contractual hit that makes it possible to get to the opportunity.

    But yes, I've never found it hard to persuade people of the value of ET, particularly if they've been doing none at all before.