Tuesday, November 29, 2011

Exploratory Testing - collected stuff

tl;dr: lots about Exploratory Testing

A colleague asked me if I had written articles or blogs about Exploratory Testing. Why yes, I have. It looks like it's time to share them more prominently.

Exploratory Testing is a crucial element that is often poorly integrated, so when I write about testing, I tend to make reference to Exploratory Testing. However, I don't particularly think of ET as the only interesting game in town, so when I write about it, I hope I put it in a relatively rational, consistent and practical testing landscape. Hence this feels like the first time I have consciously made an inventory of the stuff I've written about ET.

Here goes:

Of my long-form ET stuff, the paper that gets the most citations is 'Adventures in Session-Based Testing', which is about managing ET. I wrote it around ten years ago with the very brilliant Niel van Eeden. It won 'Best Paper' at EuroSTAR and STARWest. I updated it, so it occasionally gets called 'Further Adventures...'

The one I'm most fond of is 'Four Exercises for Teaching Exploratory Testing',* but although it went to the Workshop on Teaching Software Testing 5 back in 2006 and should have been part of the online materials, it vanished instead into online limbo. Astonishingly, when I searched just now, it's finally there - but orphaned from the rest of the site. If you can find a path to it from http://www.testingeducation.org/ I'll give you a hug.

You might already recognise the Black Box machines, which get plenty of attention, and are the single biggest cause of random strangers saying hello at conferences. Occasionally someone on a train (typically to or from Paddington) will say "aren't you... didn't you...", which is odd, but nice**. I know of a dozen or so people and organisations who use them for recruitment purposes – so let's call them my attempt at a balance to the useless five years I spent trying to nudge the ISEB exam marginally closer to fit for purpose.

Agile people tend to have come across Elisabeth Hendrickson's excellent Test Heuristics Cheat Sheet, which has my name on it. I contributed when she and I put our Exploratory Testing classes together for a (very enjoyable) 2-hander that we ran in London and in California.

There's plenty of other papers available at http://www.workroom-productions.com/papers.html, and many have related material. A Positive View of Negative Testing has more on techniques, Things Testers Miss is on bug stories, Testing in an Agile Environment covers my experiences of the fit (and friction) of testers - often using ET - on agile projects. In my most recent long-form paper, The Irrational Tester, I appropriated some fashionable ideas from behavioural economics, and I hope not only gave them testing context, but enabled more substantial exploration by drilling back from the pop science to the original research. That one got another 'Best Paper', this time from STAREast.

On that same 'papers' page, you'll find a short series of more conversational short-form things under the heading 'Exploratory Testing Notes'. They go with my Getting a Grip on Exploratory Testing workshop - they're not carefully-checked whitepapers, but nor are they short sharp blog postings. There's yet more that goes with the course, but it changes pretty much every time I do the thing.

As for blog postings, well, you're here - but Blogger is, ironically, a rotten thing for searching.
This is an entry on tools for ET http://workroomprds.blogspot.com/2008/06/tools-for-exploratory-testing.html ,
and this for assurance http://workroomprds.blogspot.com/2011/06/how-to-assure-exploratory-testing.html ,
and two together that may be of interest http://workroomprds.blogspot.com/2008_11_01_archive.html .

Reading this, it's obvious (to me) that lots of the other ET-related stuff I've written over the years has slipped away. That's the problem with the internet - ephemera are eternal, but useful stuff gets drowned. I'll fish some out and post them over the next few weeks.

Finally - I run a workshop from time to time called Getting a Grip on Exploratory Testing. It's all hands-on, and is limited to 12 people. I'm running a public class in Oxford on 25-27 January. Lots of friendly testers have twittered about it, and some (who have been on the thing) have recommended it. You'll need to look now, and book quickly, to get to the early-bird discount by the end of the week.

* Two things to note: firstly, the exercises describes are software, and available to all. Contact me and I'll send you them - plenty of people do, and they're used all over the world. Secondly, they're deep enough exercises to still be part of my workshop. By all means have a play, but if you're thinking of coming on my workshop, be aware that novelty is important to exploration and you won't get as much from the workshop. That said, I always have alternatives available if someone turns out to be familiar with the exercises.
** Sometimes they just want me to sing a song, which is still odd, kind-of-nice, and tends to mean they're a Bulgarian. Once, someone on a train was both a tester and a Bulgarian. We had lots to talk about. I should call him and arrange lunch.


  1. This is good stuff all the way through.
    The 'Four Exercises for Teaching Exploratory Testing' is new for me, and looks very good.
    Especially the 'compare machine A and D' appeals to me.
    I'm sending a request by mail.

  2. The way to find Four Exercises is:
    1. Go to http://www.testingeducation.org/ (redirects to http://testingeducation.org/wordpress/)
    2. On the right side, click Workshops
    3. Click '2006 Workshop on the Teaching of Software Testing (WTST 5)'
    4. You get to a Not Found page, click WTST 5 2006
    5. You are on http://www.wtst.org/?page_id=18, click 'James Lyndsay'

  3. Rikard - next time I see you, you get a hug.

    Other readers - here's a five-minute tester exercise which I enjoyed: follow Rikard's instructions above, keep track of any bugs, confusions, or extra observations, and see how your exploratory instincts match (or don't match) his steps.
    * What does your exploration tell you about the site?
    * What do his choices make you think about your choices?