7+3 = 11

tl;dr – a boring trivial bug is causing me to procrastinate by writing about it
tool – a spreadsheet to help you choose input data to spot the particular pathology

I love moo cards*.

Here's a snap of a recent bill from moo. Spot the bug.

100 cards £9.17 ... Shipping £2.50 ... VAT £2.33 ... Total £14.01 ... new blog post: priceless

The problem is a known pathology**. It's not uncommon to find that basket calculations are sometimes off by a penny; the calculations are done with precision, and those precise numbers are fiddled to fit with our quantum of currency - the penny. The error fits the fiddle.

In this case, the total including delivery and VAT looks as if it should be precisely £14.004. Expecting this to be £14.00, one might be tempted to speculate that the total has been rounded up my mistake, but two things make me not so sure.

Ⴀ) I generally see problems related to truncations (which always go down; £14.004 -> £14.00) and normal rounding (£14.005-> £14.01, but £14.004-> £14.00).
Ⴁ) thinking about it, I had a 10% discount on the normal price as a sop for the knock on effects of a previous bug. Discounts add another layer of complexity.

Let's work the numbers:

• £10.19 is the normal price.
• After the 10% discount, that would be £9.171, not £9.17.
• Add £2.50 delivery to arrive at £11.671.
• 20% VAT on is £2.3342.
• The precise total is £14.0052 - which will be rounded up to £14.01.
• The VAT component is £2.3342 - which will be rounded down to £2.33.

That seems more plausible.

Is this a rare combination of numbers? I built a spreadsheet to explore, and it is not; 300 prices between 1p and £10 show this behaviour.

All this is in the context of a 10% discount, 20% tax and £2.50 delivery. But my spreadsheet is a model, so I can change the conditions. Playing with it gives me the following empirical understandings:

• you don't see this problem without a discount;
• within reasonable ranges, picking alternative discounts doesn't change the incidence much;
• within reasonable ranges, changing the tax doesn't change the incidence much - I've seen it go down to 200;
• the range of incidence seems to be 200-300 for 'reasonable' ranges of tax and discount
• the delivery charge doesn't matter if it's to 2dp (and my model is inaccurate with 3dp)

Constraining myself to a basket with one item, I expect that I can sit down and demonstrate mathematically to my own satisfaction that in order to see a total that rounds up (ie £14.005), and an associated tax that rounds down(ie £2.3342), you need a price with a third decimal place - ie a normal price that has already been adjusted in some way. But that efficiency, while attractive, is a procrastination too far. For now, I'm happy with the general rule of thumb; you only see this problem when at least one thing in your basket can have a price that includes fractions of a penny - but if the potential is there, you'll see if for 20-30% of your possible prices.

Coders: One solution is do all calculations off-screen to full precision, but produce the totals on the bill from the numbers that actually go on the bill. Another is to round your total to 2dp before calculating tax. Of course this can mean having two containers for very similar information.

So far, so fun. For testers.

Frankly, I don't mind paying the extra penny. My problem is what the penny does to my paperwork.

I'm doing my VAT accounts, where I separate the £2.33 from the rest of the total. Moo's fluff on their bill means that stuff that should add up to zero, doesn't. I'll have to fudge the penny, which means introducing a special case. I'll have to be careful, because special cases are where I make accounting mistakes. That's a pain. I hope that you (or Moo) can use that description to advocate a fix for similar bugs.

And I hope that you go out there to find them. Here's the link to that spreadsheet again. I'll use it to generate data to help me reveal this issue***. You may use it and abuse as you wish. Please attribute me if you use it in public. It's got a second page that shows incidence, and a third with instructions, license and known bugs.

*  Those of you who have had a business card from me are charmed by them, too. Moo's custom postcards will lend excellent grooviness to a game I have in mind. I want to make special stickers for a bunch of post-it related activities. Moo have always responded swiftly and sweetly to problems, and  to top it all, they're local.

** I know this pathology, and I look for it when I test. Indeed, I've got an exercise based on something very similar in one of my classes. Some people question the veracity of that exercise; surely no-one really has obvious errors like this any more. Ha.

*** I tried a google docs version, but it runs like a three legged dog on Safari and Firefox. I was so discouraged I didn't bother with Chrome...

Something for the Weekend? 002

Briefly – I'm back in the studio for the weekend* – I'm fascinated by the way that technology enables interactive art. Exploration, discovery and emergent properties are desirable, even crucial qualities of the work.

Here are the sites of two people whose work I find especially interesting:

Brendan Dawes – and you should also try MagneticNorth
Robert Hodgin – who has more at his blog, Flight404

I'm thinking of getting to know Processing, before trying to get to grips with Cinder. Any of you got experience to share?

Enjoy the weekend.



* teaboy, mainly

Broken by design

tl;dr – I can't print or save a filled-in form generated by  HMRC software

A pet peeve, involving the Taxman and Adobe.

The Taxmen need a form. They'd like it online, and generously supply free software to help me get the numbers in the boxes. I use their software. It produces a form as a .pdf.

It's a dynamically-filled form*, so if I use one of my usual pdf readers, the boxes are devoid of numbers. Only with Adobe's reader can I see my numbers in the boxes. Adobe's reader is desperately slow and buggy, and I need to explicitly allow it to trust this locally-made form in order to see anything meaningful – but that's not my peeve. My peeve starts when I get to a point where the form is useful, and I'm shown a neat purple message:

        You cannot save data typed into this form. Please print your completed form if you would like a copy for your records.

Well, I would. Note that I've not typed any data into the form; it's been generated for me by the Taxman's tool. I go to print the form. I tend to print to .pdf, as I'm swamped with archived paper as it is, and a .pdf is both searchable and findable. A dialog appears, jauntily sporting the following:

        Saving a PDF file when printing is not supported. Instead, choose File > Save.

I consider printing it to paper**, scanning it in, OCR-ing the thing and calling it quits. Just in case, I try File > Save. No one will be surprised to know that I'm told:

        Data typed into this form will not be saved. Adobe Reader can only save a blank copy of this form.

A blank form? I'm sure that's what the taxman intended. The observant will notice that, as happens so often, following the instructions will put me into a self-defeating infinite loop. I've met this before, and that's my peeve.

It's big guns time. I pull out Acrobat 6 Professional. We're into software-that-costs-money territory here, and indeed have plunged straight into that unhappy valley of software-that-I-need-once-in-a-blue-moon-but-buggers-up-my-machine-to-such-an-extent-that-I-wince. Acrobat Professional is, for those of you unacquainted with Adobe's upgrade paths, ongoingly expensive. It also plays nasty with the other children in the sandpit, and doesn't do anything (except this) that I need.

A minute or two later, after it has managed to load, trashed the screen redraw, bunged the CPU to 100% and asked me to upgrade (not on your nelly, you eight-year-old, tired, hack, although I admit I have considered it), I try printing again.

        Saving a PDF file when printing is not supported. Instead, choose Save from the File menu.

There's that 'not supported' message again. Acrobat aside, I've not yet met an application that can print, but that can't aim it at a pdf. Perhaps I should set up a .pdf printer - but choosing not to address the bristles on that yak for the moment, I choose Save from the File menu, and - astonishingly - I can.


I suspect that by not supported, Adobe actually means restricted to the paid-for version. I suspect (suspicious tester that I am) that Adobe have done this on purpose. The taxman has chosen to provide me with a tool that throws my data away, unless I pay Adobe for the joy of keeping it. I wonder  whether the Taxman intended, condoned, or just didn't notice this behaviour.


Post scriptum***: As it happens, the tool turns out to be a dead end. The unprintable form is for my records only. Once I'm done slapping the desk, I fill in the online form in seconds and I'm done.



* For the initiated, this means that the .pdf (empty, pretty) is accompanied by a .fdf (just the numbers).
** Portable Document Format? My arse. Portable when folded up and shoved in a briefcase.
*** As distinct from PostScript. Print joke. Ah ha ha ha, bonk****.
**** Man laughing his head off.

Something for the Weekend? 001 (zero-padded in hope)

Wil Shipley writes code; I use tools he has had a hand in* at least weekly, more so when I’m onsite**. He seems to be an auteur, involved in all stages of translating ideas into code into cash. He also writes words – copiously, but no longer regularly as far as his blog is concerned. A few years ago, he wrote up a narrative describing his thought processes and discoveries as he worked through a rotten bug. It’s called The Greatest Bug of All, and is packed with meaty goodness.

For those of you who are more visual, those who are interested in variation and multiples, or those fascinated by the anonymous human touch, here is Stephen Wragg’s collection of walking men. Note the specification.

Enjoy.




* typically OmniOutliner and OmniGraffle, although Shipley has moved on since to focus on Delicious Library, which I don’t use so actively.
** If I’m allowed to use my own kit…

How to Assure Exploratory Testing

Over on LinkedIn, Paul Gerrard has started a "Test Assurance" group, and has asked "How can we assure exploratory testing?"

Lacking in discipline, I found it hard to read every bit of blither posted, but found it easy to respond. In the spirit of reuse, here is my take...

In the small sample of organisations that I’ve met who have an assurance group, each has gone about that work differently*. Some want to see that the team isn't wasting its time, some want to see that the organisation isn't developing false confidence, some want to see that the organisation can substantiate its claims, some want to see that set processes are being followed. If you're asked to assure exploratory testing, it's a good idea to find out what you are expected to judge and influence; those expectations may be in opposition to your own.

However, let’s assume that I am being asked, and I'm being asked to assure exploratory testing in an organisation that isn't doing something I find outrageous**, and I'm being asked to set up some sort of assurance without following the rails of a pre-existing culture. We can all dream.

In that case, I would work in a way that expected 'assurance' to independently assess the degree to which information coming out of a process of work could be trusted, and the degree to which the organisation as a whole trusts it. I’d expect assurance to be a sampling activity, with access to anything but without expectations of touching everything.

For exploratory testing, I would hope to:
- Watch individual exploratory testing activities to judge whether execution was skilled (I’d want a wide variety of business and testing skills on display)
- Watch the team to judge whether their exploratory testing work was supported with tools and information (exploratory testing without tools is weak and slow, exploratory testing in the dark is crippled)
- Gauge whether the team had independence of thought, and to what degree that independence was enabled and encouraged by the wider organisation (bias informs me about trust)
- Read some of the output (reports, notes, bugs) and watch some debriefs (if any) to judge how well the team transmits knowledge about its testing activities.
- Follow unexpected information to see to what extent it was valued or discounted (exploratory testing finds surprises; is that information useful and treated as such?)

I’d hope to do the following less-ET-specific tasks, too.
- Dig into any points where information could be restricted or censored (ie inappropriate sign-off, slow processing, disbelief or denial)
- Observe the use and integration of the team’s information in the wider organisation to judge whether the work was relevant, and accurately understood
- Judge the team’s sense of direction by observing the ways that information found, lessons learned, and feedback from the organisation affect the team’s longer-term choices.

I hope that our hypothetical organisation would use these insights to help jiggle whatever needed jiggling, and that once jiggled, the organisation could feel that they could trust the information from the team even more, and that the team would feel even more relevant and valued. Then I'd kick myself for signing the NDA that stopped me writing about it.



* Some have made the system, some have bought it, some have commissioned it, some have bolted it together. None have used exploratory testing as their sole means of testing. And I've never needed to 'sell' exploratory testing to those organisations that are risk-aware enough to have an assurance group.
** like trying to find a way to avoid clear responsibilities in a legally-defensible way, or trying to avoid the truth about the system under test

Testing, testing, 1 2 3

Tfft... dffd... is this thing on?*

I'm spending a lot of time, recently, in studios. Some of them not my own. Here are two patterns of behaviour:

А: Silence - "Can you hear me?" - "Yup" - cue playback / recording until finished or otherwise interrupted.
Б: Noises (multiple, some questionable) - "What's that?" - "What?" - "That..." - "Oh. Um..." - stop recording, fiddle until the unexpected is understood.

Hello, world.

* Blogger stopped letting me update the server, the server went pear shaped, and I (copiously) lost enthusiasm. None of these things were connected, but their effect in combination produced a long period of silence. Let's see if this is an intermittent burble, or a gradually-increasing stream.

Has the UK SIGiST lost its way?

The SIGiST used to be the go-to testing gathering in the South East, if not across the country. 

The meetings had genuine energy, the speakers were world-class (and from all over the world), and SIGiST-connected movements led towards successful (in numbers if not acceptance) standards and exams. The group was one of the largest and most active Special Interest groups in the BCS.

Over the last few years, it has come more firmly under the wing of the BCS. It has lost its website, and with the website has gone any degree of facilitated online interaction between attendees. Participants have been disenfranchised; firstly in losing their vote (you now have to be a BCS member to vote), and secondly by the extended term limits of the officers.

In the few years since these changes, London tester gatherings like the TMF and the Tester Meetup have risen in frequency and attendance, SkillsMatter days have taken off with a range of testing themes, and XTC and ACCU regulars extend a warm and knowing welcome to testers. There is constant twitter chatter. Here on the STC, the online community is fertile ground for newspapers, books, groups and ideas. 

It's two days to the SIGiST. The program for the event was announced on 24 August for the 16 September - less than four weeks before. The last post on the SIGiST's facebook wall was May. There's no LinkedIn event. Whack SIGiST into twitter, and all you get are references to the Brisbane SIGiST (let's ignore @mpkhosla reminding @MarkCTest that it exists, and my own plaintive self-promotional tweet). Until I set it up twenty minutes ago, there was no listing for the SIGiST as an event on the STC. There are still spaces available. 

At this late stage, one would not expect to get time off work, or funding. However, there is a single overriding reason to go, if you care about this once-fine forum: the meeting on Thursday is the AGM. 

If you're a BCS member (I'm not), and can spare the time, and want to say something, then please get yourself to the RCoG on Thursday morning and wake the place up. I've got the slot straight after the AGM - I'm more than happy to adjust my talk if you have something (anything) crucial to say. Note: You don't have to pay the entrance fee to attend the AGM. 

If you're on the SIGiST committee, and are blowing sparks because I've written this here and on STC, then I can't tell you how relieved I am that you're reading.

Cheers - 

James

The Irrational Tester - video and new version of paper

Here is version 1.06 of my paper "The Irrational Tester". SQE have kindly posted a video of (one version of) the talk that goes with it; keynote at STARWest 2009.

TestLab stuff coming soon...

TestLab Followup - not yet...

Bart Knaack and I ran the "TestLab" at EuroSTAR. I've got lots to write about it - but unfortunately (or fortunately) I came back to an unexpected job in the UK. I'll post conclusions, bugs, pictures and more in a day or two. I've not yet submitted the bugs to OpenEMR, either. It will all happen, but not immediately!

For now, read Michael Bolton's blog postings Bravo! and Best Bug, or Rikard Edgren's Notes.

Cheers -

James

Oh for goodness sake

The estimable Association for Software Testing isn't exactly approachable via Google, is it?



As it happens, it's reasonable on Bing.

The AST is an association for testers, run by testers. Perhaps it works on their machine.

I shouldn't be so sarky. I've sponsored their conference before now, in 2006 and 2008. I recommend you go.

(now, let's see if my journal-to-blog tool manages picture attachments properly. Expect to see an edit if it's still bust.)

Exploratory Testers - London 3 July - Prince Arthur, Euston.

As you may know from a previous posting, Michael Bolton and I are going to bring the participants of our separate Exploratory Testing / Rapid Testing classes together, after our classes finish on Friday evening. Please join us!

Date:                Friday 3 July, from 5:30 pm
Place:                The Prince Arthur pub, near Euston station. Reviews from Beer in the Evening and Fancy a Pint. Here's Google streetview, which should help you get there. Being just next to Euston, it's good for tubes, trains, bicycles and other rational London transport.

I think this is a remarkable opportunity – I hope to see you there!

Cheers - James

London, 3 July – Exploratory testers in a pub

Tentative plan: Michael Bolton and I are (separately) teaching in London in July. Our courses, mine on Exploratory Testing, his on Rapid Testing, end on the same day. We've talked about bringing the participants together in a London pub at the end of the course. Yesterday, at the SIGiST, we slipped into an announcement.

We don't yet have a pub set, and one or both classes will have to travel (my course is in the City, Mike's is in Westminster). We hope that we will have an enthusiastic, engaged, but not-too-exhausted group of exploratory/rapid testers, and that by bringing the participants in our classes together, we'll all have a lovely time, and learn some stuff.

Details will be posted closer to time, but are, so far: A group of exploratory testers will be meeting in London sometime in the early evening of Friday 3 July. It will be a unique event. Please join us.

PS - a few places are still available on my course. Details here, book here.