Something for the Weekend? 004 - Games with unknown rules

tl;dr - some games don't have fixed or known rules. Go here or here.

A long time ago, in a wooden house on the shoulder of a snowcovered mountain, I found myself sitting in a circle of people I didn't know at all. We were idly tossing a ball about. The ball was a complex thing with lights and buttons and fresh batteries – and as it moved from person to person, we shared arbitrary rules that were immediately forgotten. We noticed we had the attention of a couple sitting outside the group. They were, it emerged, players in the UK's Go team, and they were simply fascinated.

We continued to throw the ball. Occasionally, another rule would turn up. One of the Go players objected that our rules were inconsistent. Rising to the bait, we gently corrected him by explaining a hitherto unappreciated subtlety to the ruleset. More Go players were drawn towards the circle. As the Go players got down to organising and double-checking, our new rules explored variables beyond buttons and lights; the pause between throws, how hard the chuck, where the target was looking, whose friend they were, whether their name started with a vowel, whether a previous in-game action had temporarily changed their name...

This was, of course, nerd-sniping. But, more interestingly, it was an unexpected kind of game; the kind of game where the rules change. The Go players, on a team trip and consequently pretty much only in contact with other Go players, were more-than-usually locked into a pattern where rules were constant. Believing that all games have fixed rules is an easy habit to fall into.

Nevertheless, games with variable rules aren't unusual – I've spent happy hours playing bar chess, word disassociation, and parroting Mornington Crescent. If you play any games with a five year old, you'll know that rules are (a) important and (b) made up on the spot.

It's tempting to see the world as a game. Some people who explain the world use games correspondingly; as metaphors for the real world. The trouble is that games with fixed, agreed and finite rules are not always a great model. It's all very well to count heads and tails, but we forget that sometimes the coin falls in our tea and the resulting dousing trashes the laptop on which we're attempting to keep score.

The rules of our world are generally local, temporary and inconsistent. Very few rules are universal, very few activities not involving time, energy and accountancy are zero-sum. For me, the joy and value of maths, physics, music, cookery and coding is the discovery of rules; these deeper, emergent, unexpected truths. It's not about playing by the rules, but playing with. So, if you're going to play a game as a metaphor for life, you might consider using a game where the rules are on the unreasonable side of realistic.

All of which is a long and late introduction to: Something for the Weekend? 004 - Games with unknown rules
List of games with mutable rules
List of games with concealed rules

Enjoy.

James Bach moves very fast...

Something for the Weekend? 003

This turned up in Donella Meadows' excellent Thinking in Systems: A Primer*.
I've not seen it before, and thought you'd enjoy it.

A system is a big black box
Of which we can’t unlock the locks,
And all we can find out about
Is what goes in and what comes out.

by Kenneth Boulding

...there's more, of course. But I'm not convinced that posting the whole thing is a fair (ab)use of copyright. You'll find it on less fussy people's sites. Go fish.

* That's an Amazon affiliate link... trying it out.

7+3 = 11

tl;dr – a boring trivial bug is causing me to procrastinate by writing about it
tool – a spreadsheet to help you choose input data to spot the particular pathology

I love moo cards*.

Here's a snap of a recent bill from moo. Spot the bug.

100 cards £9.17 ... Shipping £2.50 ... VAT £2.33 ... Total £14.01 ... new blog post: priceless

The problem is a known pathology**. It's not uncommon to find that basket calculations are sometimes off by a penny; the calculations are done with precision, and those precise numbers are fiddled to fit with our quantum of currency - the penny. The error fits the fiddle.

In this case, the total including delivery and VAT looks as if it should be precisely £14.004. Expecting this to be £14.00, one might be tempted to speculate that the total has been rounded up my mistake, but two things make me not so sure.

Ⴀ) I generally see problems related to truncations (which always go down; £14.004 -> £14.00) and normal rounding (£14.005-> £14.01, but £14.004-> £14.00).
Ⴁ) thinking about it, I had a 10% discount on the normal price as a sop for the knock on effects of a previous bug. Discounts add another layer of complexity.

Let's work the numbers:

• £10.19 is the normal price.
• After the 10% discount, that would be £9.171, not £9.17.
• Add £2.50 delivery to arrive at £11.671.
• 20% VAT on is £2.3342.
• The precise total is £14.0052 - which will be rounded up to £14.01.
• The VAT component is £2.3342 - which will be rounded down to £2.33.

That seems more plausible.

Is this a rare combination of numbers? I built a spreadsheet to explore, and it is not; 300 prices between 1p and £10 show this behaviour.

All this is in the context of a 10% discount, 20% tax and £2.50 delivery. But my spreadsheet is a model, so I can change the conditions. Playing with it gives me the following empirical understandings:

• you don't see this problem without a discount;
• within reasonable ranges, picking alternative discounts doesn't change the incidence much;
• within reasonable ranges, changing the tax doesn't change the incidence much - I've seen it go down to 200;
• the range of incidence seems to be 200-300 for 'reasonable' ranges of tax and discount
• the delivery charge doesn't matter if it's to 2dp (and my model is inaccurate with 3dp)

Constraining myself to a basket with one item, I expect that I can sit down and demonstrate mathematically to my own satisfaction that in order to see a total that rounds up (ie £14.005), and an associated tax that rounds down(ie £2.3342), you need a price with a third decimal place - ie a normal price that has already been adjusted in some way. But that efficiency, while attractive, is a procrastination too far. For now, I'm happy with the general rule of thumb; you only see this problem when at least one thing in your basket can have a price that includes fractions of a penny - but if the potential is there, you'll see if for 20-30% of your possible prices.

Coders: One solution is do all calculations off-screen to full precision, but produce the totals on the bill from the numbers that actually go on the bill. Another is to round your total to 2dp before calculating tax. Of course this can mean having two containers for very similar information.

So far, so fun. For testers.

Frankly, I don't mind paying the extra penny. My problem is what the penny does to my paperwork.

I'm doing my VAT accounts, where I separate the £2.33 from the rest of the total. Moo's fluff on their bill means that stuff that should add up to zero, doesn't. I'll have to fudge the penny, which means introducing a special case. I'll have to be careful, because special cases are where I make accounting mistakes. That's a pain. I hope that you (or Moo) can use that description to advocate a fix for similar bugs.

And I hope that you go out there to find them. Here's the link to that spreadsheet again. I'll use it to generate data to help me reveal this issue***. You may use it and abuse as you wish. Please attribute me if you use it in public. It's got a second page that shows incidence, and a third with instructions, license and known bugs.

*  Those of you who have had a business card from me are charmed by them, too. Moo's custom postcards will lend excellent grooviness to a game I have in mind. I want to make special stickers for a bunch of post-it related activities. Moo have always responded swiftly and sweetly to problems, and  to top it all, they're local.

** I know this pathology, and I look for it when I test. Indeed, I've got an exercise based on something very similar in one of my classes. Some people question the veracity of that exercise; surely no-one really has obvious errors like this any more. Ha.

*** I tried a google docs version, but it runs like a three legged dog on Safari and Firefox. I was so discouraged I didn't bother with Chrome...

Something for the Weekend? 002

Briefly – I'm back in the studio for the weekend* – I'm fascinated by the way that technology enables interactive art. Exploration, discovery and emergent properties are desirable, even crucial qualities of the work.

Here are the sites of two people whose work I find especially interesting:

Brendan Dawes – and you should also try MagneticNorth
Robert Hodgin – who has more at his blog, Flight404

I'm thinking of getting to know Processing, before trying to get to grips with Cinder. Any of you got experience to share?

Enjoy the weekend.



* teaboy, mainly

Broken by design

tl;dr – I can't print or save a filled-in form generated by  HMRC software

A pet peeve, involving the Taxman and Adobe.

The Taxmen need a form. They'd like it online, and generously supply free software to help me get the numbers in the boxes. I use their software. It produces a form as a .pdf.

It's a dynamically-filled form*, so if I use one of my usual pdf readers, the boxes are devoid of numbers. Only with Adobe's reader can I see my numbers in the boxes. Adobe's reader is desperately slow and buggy, and I need to explicitly allow it to trust this locally-made form in order to see anything meaningful – but that's not my peeve. My peeve starts when I get to a point where the form is useful, and I'm shown a neat purple message:

        You cannot save data typed into this form. Please print your completed form if you would like a copy for your records.

Well, I would. Note that I've not typed any data into the form; it's been generated for me by the Taxman's tool. I go to print the form. I tend to print to .pdf, as I'm swamped with archived paper as it is, and a .pdf is both searchable and findable. A dialog appears, jauntily sporting the following:

        Saving a PDF file when printing is not supported. Instead, choose File > Save.

I consider printing it to paper**, scanning it in, OCR-ing the thing and calling it quits. Just in case, I try File > Save. No one will be surprised to know that I'm told:

        Data typed into this form will not be saved. Adobe Reader can only save a blank copy of this form.

A blank form? I'm sure that's what the taxman intended. The observant will notice that, as happens so often, following the instructions will put me into a self-defeating infinite loop. I've met this before, and that's my peeve.

It's big guns time. I pull out Acrobat 6 Professional. We're into software-that-costs-money territory here, and indeed have plunged straight into that unhappy valley of software-that-I-need-once-in-a-blue-moon-but-buggers-up-my-machine-to-such-an-extent-that-I-wince. Acrobat Professional is, for those of you unacquainted with Adobe's upgrade paths, ongoingly expensive. It also plays nasty with the other children in the sandpit, and doesn't do anything (except this) that I need.

A minute or two later, after it has managed to load, trashed the screen redraw, bunged the CPU to 100% and asked me to upgrade (not on your nelly, you eight-year-old, tired, hack, although I admit I have considered it), I try printing again.

        Saving a PDF file when printing is not supported. Instead, choose Save from the File menu.

There's that 'not supported' message again. Acrobat aside, I've not yet met an application that can print, but that can't aim it at a pdf. Perhaps I should set up a .pdf printer - but choosing not to address the bristles on that yak for the moment, I choose Save from the File menu, and - astonishingly - I can.


I suspect that by not supported, Adobe actually means restricted to the paid-for version. I suspect (suspicious tester that I am) that Adobe have done this on purpose. The taxman has chosen to provide me with a tool that throws my data away, unless I pay Adobe for the joy of keeping it. I wonder  whether the Taxman intended, condoned, or just didn't notice this behaviour.


Post scriptum***: As it happens, the tool turns out to be a dead end. The unprintable form is for my records only. Once I'm done slapping the desk, I fill in the online form in seconds and I'm done.



* For the initiated, this means that the .pdf (empty, pretty) is accompanied by a .fdf (just the numbers).
** Portable Document Format? My arse. Portable when folded up and shoved in a briefcase.
*** As distinct from PostScript. Print joke. Ah ha ha ha, bonk****.
**** Man laughing his head off.

Something for the Weekend? 001 (zero-padded in hope)

Wil Shipley writes code; I use tools he has had a hand in* at least weekly, more so when I’m onsite**. He seems to be an auteur, involved in all stages of translating ideas into code into cash. He also writes words – copiously, but no longer regularly as far as his blog is concerned. A few years ago, he wrote up a narrative describing his thought processes and discoveries as he worked through a rotten bug. It’s called The Greatest Bug of All, and is packed with meaty goodness.

For those of you who are more visual, those who are interested in variation and multiples, or those fascinated by the anonymous human touch, here is Stephen Wragg’s collection of walking men. Note the specification.

Enjoy.




* typically OmniOutliner and OmniGraffle, although Shipley has moved on since to focus on Delicious Library, which I don’t use so actively.
** If I’m allowed to use my own kit…

How to Assure Exploratory Testing

Over on LinkedIn, Paul Gerrard has started a "Test Assurance" group, and has asked "How can we assure exploratory testing?"

Lacking in discipline, I found it hard to read every bit of blither posted, but found it easy to respond. In the spirit of reuse, here is my take...

In the small sample of organisations that I’ve met who have an assurance group, each has gone about that work differently*. Some want to see that the team isn't wasting its time, some want to see that the organisation isn't developing false confidence, some want to see that the organisation can substantiate its claims, some want to see that set processes are being followed. If you're asked to assure exploratory testing, it's a good idea to find out what you are expected to judge and influence; those expectations may be in opposition to your own.

However, let’s assume that I am being asked, and I'm being asked to assure exploratory testing in an organisation that isn't doing something I find outrageous**, and I'm being asked to set up some sort of assurance without following the rails of a pre-existing culture. We can all dream.

In that case, I would work in a way that expected 'assurance' to independently assess the degree to which information coming out of a process of work could be trusted, and the degree to which the organisation as a whole trusts it. I’d expect assurance to be a sampling activity, with access to anything but without expectations of touching everything.

For exploratory testing, I would hope to:
- Watch individual exploratory testing activities to judge whether execution was skilled (I’d want a wide variety of business and testing skills on display)
- Watch the team to judge whether their exploratory testing work was supported with tools and information (exploratory testing without tools is weak and slow, exploratory testing in the dark is crippled)
- Gauge whether the team had independence of thought, and to what degree that independence was enabled and encouraged by the wider organisation (bias informs me about trust)
- Read some of the output (reports, notes, bugs) and watch some debriefs (if any) to judge how well the team transmits knowledge about its testing activities.
- Follow unexpected information to see to what extent it was valued or discounted (exploratory testing finds surprises; is that information useful and treated as such?)

I’d hope to do the following less-ET-specific tasks, too.
- Dig into any points where information could be restricted or censored (ie inappropriate sign-off, slow processing, disbelief or denial)
- Observe the use and integration of the team’s information in the wider organisation to judge whether the work was relevant, and accurately understood
- Judge the team’s sense of direction by observing the ways that information found, lessons learned, and feedback from the organisation affect the team’s longer-term choices.

I hope that our hypothetical organisation would use these insights to help jiggle whatever needed jiggling, and that once jiggled, the organisation could feel that they could trust the information from the team even more, and that the team would feel even more relevant and valued. Then I'd kick myself for signing the NDA that stopped me writing about it.



* Some have made the system, some have bought it, some have commissioned it, some have bolted it together. None have used exploratory testing as their sole means of testing. And I've never needed to 'sell' exploratory testing to those organisations that are risk-aware enough to have an assurance group.
** like trying to find a way to avoid clear responsibilities in a legally-defensible way, or trying to avoid the truth about the system under test

Testing, testing, 1 2 3

Tfft... dffd... is this thing on?*

I'm spending a lot of time, recently, in studios. Some of them not my own. Here are two patterns of behaviour:

А: Silence - "Can you hear me?" - "Yup" - cue playback / recording until finished or otherwise interrupted.
Б: Noises (multiple, some questionable) - "What's that?" - "What?" - "That..." - "Oh. Um..." - stop recording, fiddle until the unexpected is understood.

Hello, world.

* Blogger stopped letting me update the server, the server went pear shaped, and I (copiously) lost enthusiasm. None of these things were connected, but their effect in combination produced a long period of silence. Let's see if this is an intermittent burble, or a gradually-increasing stream.

Has the UK SIGiST lost its way?

The SIGiST used to be the go-to testing gathering in the South East, if not across the country. 

The meetings had genuine energy, the speakers were world-class (and from all over the world), and SIGiST-connected movements led towards successful (in numbers if not acceptance) standards and exams. The group was one of the largest and most active Special Interest groups in the BCS.

Over the last few years, it has come more firmly under the wing of the BCS. It has lost its website, and with the website has gone any degree of facilitated online interaction between attendees. Participants have been disenfranchised; firstly in losing their vote (you now have to be a BCS member to vote), and secondly by the extended term limits of the officers.

In the few years since these changes, London tester gatherings like the TMF and the Tester Meetup have risen in frequency and attendance, SkillsMatter days have taken off with a range of testing themes, and XTC and ACCU regulars extend a warm and knowing welcome to testers. There is constant twitter chatter. Here on the STC, the online community is fertile ground for newspapers, books, groups and ideas. 

It's two days to the SIGiST. The program for the event was announced on 24 August for the 16 September - less than four weeks before. The last post on the SIGiST's facebook wall was May. There's no LinkedIn event. Whack SIGiST into twitter, and all you get are references to the Brisbane SIGiST (let's ignore @mpkhosla reminding @MarkCTest that it exists, and my own plaintive self-promotional tweet). Until I set it up twenty minutes ago, there was no listing for the SIGiST as an event on the STC. There are still spaces available. 

At this late stage, one would not expect to get time off work, or funding. However, there is a single overriding reason to go, if you care about this once-fine forum: the meeting on Thursday is the AGM. 

If you're a BCS member (I'm not), and can spare the time, and want to say something, then please get yourself to the RCoG on Thursday morning and wake the place up. I've got the slot straight after the AGM - I'm more than happy to adjust my talk if you have something (anything) crucial to say. Note: You don't have to pay the entrance fee to attend the AGM. 

If you're on the SIGiST committee, and are blowing sparks because I've written this here and on STC, then I can't tell you how relieved I am that you're reading.

Cheers - 

James

The Irrational Tester - video and new version of paper

Here is version 1.06 of my paper "The Irrational Tester". SQE have kindly posted a video of (one version of) the talk that goes with it; keynote at STARWest 2009.

TestLab stuff coming soon...

TestLab Followup - not yet...

Bart Knaack and I ran the "TestLab" at EuroSTAR. I've got lots to write about it - but unfortunately (or fortunately) I came back to an unexpected job in the UK. I'll post conclusions, bugs, pictures and more in a day or two. I've not yet submitted the bugs to OpenEMR, either. It will all happen, but not immediately!

For now, read Michael Bolton's blog postings Bravo! and Best Bug, or Rikard Edgren's Notes.

Cheers -

James