Tuesday, May 13, 2008
Heathrow T5 and software testers
However, if you've been following the story, you'll be interested to know that Huub van der Wouden and Roger Derksen from Transfer Solutions in the Netherlands will be giving the opening keynote on Testing the Heathrow Terminal 5 Baggage Handling System at the London SIGiST on June 18th.
Saturday, May 10, 2008
Searchable library - content, not title
I find that I already rent the online, searchable edition. If it wasn't such a pig to use, I'd love Safari books online.
I'd like to have an online search for my existing luggable library, to let me find what I need in books where I don't have an index, or where the index is rubbish. Google Books gets close, but only gives one return per book. If you know how to get more than one, do let me know.
Thursday, May 01, 2008
How much TV does a Wikipedia cost?
However, the interconnected world allows us all to read / see what the book's about. Here's a fine blog post of his summarising talk. For those more visually/aurally oriented, here's a the talk itself.
Answer to the question above: Clay says that each year, the people in the USA spend 2,000 times more time than has been needed, so far, to construct Wikipedia. Except that he says it better. Read the article.
A question for you: Do you know anyone who actually understands the numbers, ie can apply them in their everyday lives? Who understands fundamentally how large numbers of people and small commitments/risks/expenditures actually add up? It doesn't exactly come up in conversation much, but I don't think I know anyone who has a clear feel for this. Perhaps it's my generation. Then again, I know plenty of people who have a fine handle on atomic measurements and cosmological time - perhaps I should get to know some social scientists.
Hmm. Let's get back on theme. For the testers among you, another question: How are we sizing our beta tests? At what point might a beta test outweigh a local test team? At what point might a beta test be reasonable expected to have found problems that could surface in the first month of general use?
Wednesday, April 30, 2008
Fixing 7digital's playback problems in iTunes
Summary: you'll need to rename .mp4 to .m4a
I just bought The Age of The Understatement by The Last Shadow Puppets, which is rather lovely. As an experiment, I bought it from 7digital - it cost a fiver rather than £7.99 on iTunes.
The beggar doesn't play properly on my kit. Each track pauses for 5 seconds or so, just under 10 seconds into the track - which is hardly conducive to great listening pleasure. As far as maintaining the atmosphere goes, it's a bit like turning the house lights on in a nightclub for a few seconds, just as every track gets going.
This is not normal, so for the configurators among you, I'll describe normal. Music comes from the iTunes store or CDs. I play the songs using iTunes under Leopard on my non-intel Mac Mini. As a tester, I've got CPU monitoring on, always, and as a tester I notice that the machine is pushing as hard as it can during the gap, but not either side. This is also not normal.
Let me also describe the differences - comparing specifically with iTunes Plus, as both deliver AAC audio without DRM protection.
The most obvious difference is that 7digital sent me the files encoded at 320kbps, compared with iTunes' 256kbps. The problem could be to do with the way iTunes codec responds to higher than expected encoding rates. However, I've successfully worked with AAC of my own projects encoded at this rate.
One consequence of this difference is that the files are larger, so perhaps this is at the root of my problem. However, I'm instinctively less than keen on the this as a target for investigation. Digging into that instinct, I can rationalise that 1) the kit is perfectly capable of shifting the quantity of data, 2) the problem is limited to part of a song, 3) the position and duration of the dropout seems unrelated to song length. It doesn't feel like something to do with amount of data.
Another difference: while looking at iTunes' information for the song, I find that it's listed as an MPEG-4 video file. This seems inappropriate - I'd expect it to be an AAC audio file. This is, after all, how it was described at download. I'm aware that this is linked to the extension, and checking the files, find the extension is .mp4. Pretty much everything else is .m4a, including those iTunes Plus files. Other iTunes files are .m4p. Both these extensions indicate Apple's AAC format, one unprotected, the other rights-managed. Those in the know can chastise me for (at least) two reasons. Firstly, MPEG-4 is a container, not a format. Secondly, iTunes should treat .mp4, .m4a and .m4p files the same - if it doesn't, it's iTunes that has the bug, not 7digital.
In the spirit of investigation and experimentation, I change the extension from .mp4 to .m4a, and the problem is gone - no hot-running CPU, the right details in the information. Most importantly, the music now plays without interruption.
Now, this is a hack. It may have corrected a simple mis-configuration. However, sound files are complex containers, and their interactions with various players are not always predictable. Will iTunes be able to burn these tracks onto a one-off mix CD? Will they play nicely with the iPod? If I chuck them into Ableton's Live to mix and mash, will there be a problem? Heaven knows, until I try. Perhaps I should have stuck with their MP3 versions . . .
The immediate problem is solved. I've not resolved the bug, but I've developed an understanding of it which has led to a fix for my machine and my purposes.
Now I'm bug-hunting, here are a few more I noticed along the way. These may help you fix/get round/be prepared for stuff on your own machine.
- The transition between Black Plant and I Don't Love You Anymore is stuffed - there's a hole where there should be a transition. This isn't iTunes' infamous truncation bug, it's because it sticks a pause between tracks unless told not to. It's easily fixed - setting the album to "gapless" does the trick.
- 7digital's site, for reasons best known to itself, gives the tracks un-numbered, and in reverse order - it was only when I played the album that I noticed that due to my own finger trouble, I had two Calm like Us and no Standing Next to Me. That said, re-downloading is trivially easy and (with the server in the UK) pleasantly speedy.
- I have to trigger each download individually (dull) if I don't want to install their downloader (I don't, thankyou - and I can't anyway as it's PC only), and when done have to put the tracks into my library (whine whine whine).
Customer impressions of 7digital: they are good value, but slightly troublesome. The trouble may not be their fault, but it means I have to work to use their music on my kit. Not their fault, but I may not buy again.
Customer impressions of The Age of the Understatement: a fine album - good songs, great words. Retro and fun without too much pastiche. Epic, if you're in the mood; insert iPod and become the hero of your supermarket trolley. Better fixed than buggy - I like it enough to mean that I'd buy a working copy if I'd not managed to fix this one, but not without a niggling sense of being conned.
Update: shortly after posting this entry, I was contacted by the CTO of 7digital, who is clearly on the ball. The .mp4/.m4a thing is now with his technical people. Good to help - better to be useful.
Monday, April 14, 2008
Comedy bug
I choose to uninstall the software with an application the supplier supplies. Halfway through the install, it tells me that I should have quit out of mail, as it's uninstalling a component with which it vandalised my mail application. It quits its own uninstall and gives me a nice, although temporary log.
I quit mail, and (with a hint of a tester premonition) head off to find the uninstaller, so I can uninstall the final component of this rotten puzzle.
The uninstaller has, naturally, uninstalled itself.
Marvellous.
(for those of you unfamiliar with why this is a problem, it's broadly analogous to locking the keys inside the car)
Wednesday, April 09, 2008
Running LEWT
a simple recipe (translated by Linda Schenck) neatly encapsulates and codifies the principles that I try to use for LEWT - and, indeed, extends them.
To borrow Rikard's summary, they include;
- generosity
- a sense of community
- qualifications
- cultural diversity
- trust and tolerance
- equality
- curiosity
- freedom of spirit
- small scale
It's interesting that the facile quality 'communication' is not included. It's also interesting that qualifications (which don't particularly play their part in LEWT) is more subtle that I had perhaps expected. I'll be considering making some minor changes so that people are more confident in their qualification to be part of the group - although I'll be careful about affecting its diversity. My feeling that LAWST's guru-led approach is problematic is neatly caught by the note on equality.
I'll be posting a link to the article to the LEWT discussion group, and hope to start a conversation. Let me know if you'd like to be part of it.
Creativity in Software Testing
Although I've talked about this until I've bored all around me, I've never managed to write anything coherent on the subject. So I'm pleased that I came across Rikard Edgren's Where Testing Creativity Grows, as it is one of the very few papers I'm aware of that looks at this important, yet unaccountably neglected perspective.
You can find it here: http://qualtechconferences.arobis.com/upload/documents/REdgren_where_testing_creativity_grows.pdf
I encourage you to have a read. If you know of other papers on the subject, post them here.
James
Friday, May 25, 2007
Question: What testing tool will help me test an Intranet application with lots of complex calculations?
One more thing I would like to inform you that as ours is a very small company we are looking for a tool which comes under $500.00. If any OpenSource tools availble than it is well and good.
Your budget will exclude the larger commercial tools - particularly with ongoing licensing costs. I note that your company provides IT services, and you should consider whether any tool - and the skills acquired to support it - would provide ROI across the company, rather than on a single project.
An organisation's greatest investment in testing with any tool - licensed or open-source - is likely to be the cost of understanding the tool and developing+maintaining the tests. Open source is attractive not only because of its lower initial cost, but because the tools can be modified to suit your purposes. I've detailed some open-source tools below.
Although your email is more detailed than many, it doesn't address the most important question - the purpose of your testing. If you would like to discuss this, call me using skype.
For functional testing of your underlying complex calculations, your best bet might be to write a simple test harness to supply input and check output, avoiding the user interface. This approach allows you to automate testing using powerful design techniques that can go far beyond simple regression scripts, and would potentially give you rather more confidence in the face of change. Look into ideas around combinatorial testing, fuzz testing, and model-based testing.
For testing a web interface, and some functional testing of the software accessed through a browser, I would suggest watir. If by database testing, you mean testing for the purpose of assessing performance or scalability of a database, I would suggest looking at jmeter or grinder.
Watir : http://wtr.rubyforge.org/
JMeter: http://jakarta.apache.org/jmeter/
Grinder: http://grinder.sourceforge.net/
Question: How do I get into testing?
Are you actively working in IT at the moment?
i am ready to working in software testing and development organisations
Sadly, I didn't reply for a while, until politely nudged a week or so later:
Please give a good suggestion to me
"ready to work", I assume, means that you're not currently working in the industry. The issue then is how to identify organisations that need testing skills, and how to persuade them that you have the appropriate skills and experience.
It's worth considering that organisations that make software have different requirements from those that buy software. However, technical and business skills are necessary in both, and without direct working experience in testing, you will need to persuade such organisations to consider you for a position on the basis of your technical and business knowledge.
You should make a start by identifying those organisations within your area of business that buy software - and the companies that supply those systems. Once you've got a good sense of the software and systems that make up that area, you can concentrate on those which match your technological skillset.
Organisations that make software need new testers toward the end of a project, or at the point where they are customising software for a new client. They may draw their test team from the technical resources, and as a new hire, you would be expected to be able to know enough about the technical stuff to talk to the existing team, and potentially to know more about the business than many of the people you would be joining. You would typically be doing Functional Testing.
Organisations that buy software typically test the systems that they buy only if the system has been customised (or built) for them - this most often happens in large organisations buying 'enterprise' software. These organisations may draw their test teams from current business users and add testing skills. You'd be expected to know enough about the business stuff to talk to the existing team, and potentially to know more about testing and IT than many of the people you would be joining. You would typically be doing User Acceptance Testing.
Qualifications can help - in some [organisational] cultures, you would not be considered for a testing role without a testing qualification, an appropriate business qualification, security classification, or a background in the relevant technology. If you don't have any of these, concentrate on businesses that won't require one.
Put together a CV that details you're relevant experience and shows some understanding of the area you're interested in, and send them off. You can also, if lucky and charming, make great headway networking with people in the business and technical areas.
Question: Does the test strategy vary?
My question is that does the test strategy vary for both software testing applications and mobile application testing.
Or is it the same followed for all.
And also the test plan and Test matrices are same for mobile application testing or not?
Every good software test strategy I've seen has been influenced by its context. I would expect the strategies to be different - and to reflect the difference between the two contexts you give.
The plan - the list of what happens when - follows from the strategy, inasmuch as the typical purpose of a strategy is to give structure to decisions about what to plan and how to plan.
I've no clear idea what you mean by a test matrix. If I can second-guess for a moment though, I would assume that a test matrix is some combination of individual tests. Different systems may well share a subset of tests. The difference would be in the importance of those tests, and how the information they produce is understood in the context of the project as a whole.
Question: Can you give me a Test Strategy Template?
Here are some examples of the genre . . .
Can you please send me few examples and templetes for Test Strategy....
Thanks for getting in touch. Could you tell me a little more about your project and your goals?
About my project... The project is Retail management(Web-Based Application) and its new project... the development area is J2EE. Now we are in Functional Designing phase... hope after finishing the functional design spec. we are planning to come up with Test Strategy and Test Approach and Plans...
My Ultimate goal is to deliver a Quality product to my Client...
So, can you please send me few sample Test Strategy Templates for it... hope you will do needful to me...
Thanks for this information. It sounds as though you are at a good point in the project to be developing a test strategy. You may already have a workable idea of the technical constraints and potential patterns of failure that could drive aspects of your test design. However, what will be more important to your test strategy are the values that inform your test decisions, and you've not included information about what they might be.
Technical environment and business context are important to a strategy, and it might help to see a strategy for another project in the same environment. However, there are no publicly-available examples I know of that cover this particular pairing.
I've worked on products that have a fair fit with your description. The test strategies for the projects were very different, reflecting the values and priorities of the organisation. For commercial and copyright reasons, I can't supply copies of those strategies to you.
I feel strongly that a test strategy is something that should *not* be created from a template, and I don't supply test strategy templates - either for free, or during a paid interaction. I try to make my position clear here: http://www.workroom-productions.com/strategy.html . The page does include a checklist, which may serve as a stimulus for discussion within your team.
I'm sorry not to be of more help. If you would like to speak with me on this, please call me.
I am working as a Test Manager for one of the big program has teken up my company. Program contains number of projects and applications on different technologies like Java, ATG, Web Services, Data Stage, Mainframe COBOL etc... and it has nearly 250 inbound and outbound systems. Can you please provide inputs for test strategy for these kind of programs? and it will be grate if you provide any sample or template for test strategy.
Thanks for your enquiry. Your project sounds substantial, and I don't think it can be dealt with in an simple email exchange. If you would like to speak with me on this, please call me during London business hours.
Please note: I don't supply test strategy templates - either for free, or during a paid interaction. I try to make my position clear here: http://www.workroom-productions.com/strategy.html . The page does include a checklist, which may serve as a stimulus for discussion within your team.
I'm having a self study training on test strategies and going through your website was wondering if you had Knowledge of examples or references for Test strategies of payroll based systems of which I am conversant with a few. Any help will be appreciated
I can't give you any examples that are specifically tuned to payroll systems. The purpose of a system is important to a strategy, but so are the values of the organisation, the technology of that system, the power and constraints of the potential resources.
If you're working on a self-study course, you could do worse than study the test strategies used on the payroll systems you are familiar with. You are likely to find the actual strategy is very different from the stated strategy - if one even exists. You could also write two or more *different* strategies for the *same* product and context, to expose potential alternatives and underlying rules. Try to keep such strategies short; less than 3 pages or so. I have a rule-of-thumb - the higher-level the strategy, the shorter it should be.
I am a Senior software tester and working for a consultancy company called [obscured supplier]. I am trying to prepare a Test Strategy document for the client [obscured client].Is this any free consulation given by the Work room Productions based on test strategy document .Please let me know this.
Thankyou for your enquiry. Please let me know your ideas so far - and what you might require from consultancy. I notice that [obscured client] are a US-based healthcare company.
A use for my blog
However, my email address is in the public domain, and I wonder if I can't make use of a blog to communicate some of the questions I receive about software testing, and some of the answers I give. This would be in the nature of an FAQ, although perhaps the F in that acronym is misleading. I get one or two a month, and sometimes wonder what motivates a stranger to ask me for advice.
The questions that are sent to me are typically short, and seem to need a long answer. It's all too easy to put far more information in the answer than is got in the question - a sign that one may be building one's half-baked rant on sand. I try not to let myself get carried away.
Usually, one would need more details to have a decent stab at a good answer. I'll try to engage the questioner; handing out my skype details and looking forward to clarifying emails. It's rare treat when someone actually responds.
My answers are offered for free, with all the attention to quality and detail that implies - but I also see them as a puzzle, and try to write something cogent. I offer them here in a more public forum, to share ideas and invite comments.
I've not got explicit permission to use people's questions, and so I have obscured, cut and amended their words and my answers to suit. I have a small backlog of such questions. I'll post the first shortly. You're welcome to add comments of your own.