Mad as a March Hare – Musings on Rapid Software Testing and TestBash Brighton 2016

So another year has come around, and TestBash has come and gone. What was initially planned as a series of posts around TestBash and the Rapid Software Testing course didn’t come to be, due to an unforeseen technical hitch. So instead, this is my response to the events in Brighton last week. My annual pilgrimage to Brighton for TestBash had an extra few dimensions than usually just attending workshops and the coference day

Rapid Software Testing with Michael Bolton

So as I mentioned in my last post I was about to embark on the journey into Rapid Software Testing, on this occasion led by Michael Bolton. I’ve followed Michael’s (and James Bach’s) work for some time now, like a lot of testers who identify as context driven. I’ve been using many of the techniques, models and approaches to testing that RST champions for a while; mostly via learning from others, reading and applying techniques in practice. I have however not undertaken the course before now.

 

RST with Michael Bolton - March 2016

RST with Michael Bolton – March 2016

 
However the greatest impact upon my life as a tester was being able to talk and discuss the thinking around it with one of the authors themselves. It gave the the concepts, thinking and knowledge the practical grounding it needed, framing it within the discourse with Michael. A most rewarding experience. Now the unusual thing was that not only was I attending and learning at RST, but I was also facilitating on behalf of the Ministry of Testing. That meant that my primary concern was to the needs of the group and the coach. So balancing my needs and those of the group was challenging.

We discussed heuristics, oracles and models for testing in extreme depth, at least within the time allowed. We explored and practiced techniques for deep learning, exploration and test design and strategy. We would often revisit, review and tune our thinking on each topic, where Michael fed in to our learning, but responded actively to questions and responses, challenging and exploring each one in depth.

Here for example is one of the initial activities we did. A 15 min charter using the “Triangles” application. Here are my notes, capturing what I explored and discovered with my partner.

As a tester, I wanted to observe how the triangles application recorded data, inputs and outputs in the system:

We entered a range of values and inputs in to triangles.

  1. Integers
  2. Decimals
  3. Negative

We observed that a text file, triangles.txt was created by the application

Bug – the triangles.txt file is written to the folder above the application folder, so /thingstotest not /thingstotest/triangle

Bug – when we entered values into triangles, there was no feedback on the values, just the shapes.

Bug – 

SIDEA: -1

SIDEB: 2-

SIDEC: 1

TYPE: X

SIDEA.ERROR: ILLEGAL CHARACTER

SIDEB.ERROR: ILLEGAL CHARACTER

These illegal characters are not fed back to the user in UI, so they don’t know what is or isn’t an illegal value. Not enough information is presented to the user.

Now, the triangle application was one I had been familiar with for a while, thanks to a great session at the office led by Chris Simms (@kinofrost on Twitter). My familiarity was not the point here. I reviewed my notes from that session, and they were different interms of info captured. I also had more time available in a lunchtime session. We had the opportunity to dig deep and produce some good work, some good testing in both sessions, but recognising the limited time we had is a factor that will need to be applied in all testing sessions.

Triangles is a fairly simple application. You enter three variables for the size and shape, and it responds with the type of triangle and an appropriate image of the shape. It also has a log file which records the values entered, and the application responses. The depth of the groups testing belies the simplicity of the application under test. Each group found different problems and further questions to ask, there were also many overlaps in our observations. However the learning was being blue to design testing on the fly, with little or no prior information. Easy, right?

Well, no, not easy. Asking good questions is never easy. And that’s the whole point of RST, I feel. A lot of traditional testing practice expects us to read documents, write documents and write test scripts. Testing through those approaches appears to be somewhat of an afterthought. 

RST challenges us  to ask good questions. Through asking and answering good questions we develop good testing ideas, strategies and approaches. The documentation becomes the notes we take, or in our case the mind maps we captured in another session. Our advocacy and responsibility for our testing and the products of our testing are at the heart of what RST is to me.

Below are some notes from the team, captured during the course. Bearing in mind that these are incomplete and a work in progress. They do have problems of ‘translational’ and ‘transactional’ awareness between the group, the coach (Micheal) and me (the scribe). Trying to capture people’s thoughts and learning is really hard, especially in large and vocal groups. The handwriting is mine, but this is the work of the whole group.

 

RST - Heuristics and Oracles

RST – Heuristics and Oracles

  
RST - Properties of Good Bug Reports

RST – Properties of Good Bug Reports

  
RST -  PEOPLE WORKING - a mnemonic for problem reporting

RST – PEOPLE WORKING – a mnemonic for problem reporting

  
RST -    Some testing models

RST – Some testing models

 

There is a lot to digest and process from RST, probably too much to share in a busy blog post. In summary, RST was an incredible experience. It has afforded me the opportunity for me to both challenge and consolidate my existing learning, enhance my note taking and observation skills whilst testing. It’s also allowed me through facilitation to place a greater priority on the learning experience of my colleagues than that of my own. It was hugely valuable, and I would jump at the chance to do it again. I’m grateful for Rosie Sherry for letting me facilitate on behalf of Ministry of Testing (thats me being  all corporate in the red MOT T-shirt); and to Michael for his time, knowledge and insight. Many thanks!

Rapid Software Testing  Alumni - March 2016

Rapid Software Testing Alumni – March 2016

TestBash Brighton 2016 – Workshops

So on to TestBash. The workshop day has been an event that has been introduced both through demand for rich learning opportunities in the testing community,  also the Ministry of Testing’s desire to create an environment where that can happen. In the afternoon I was running my own workshop on proxy tools, which I will leave others to reflect on in public.

For my needs I like to balance the technical learning I get with more soft skills. It’s an area I have huge problems with. Technical learning is often a case of broad reading, practice and being open to developing the skills required. My personal route to developing a reputation as a ‘security expert’ has meant that I’ve had to focus hard on the technical skills, rather than developing other elements of what testing (and software development) can be.

Christina Ohanian (@ctohanian) and Nicola Sedgwick (@nicolasedgwick) ran their workshop “Connecting the Dots: Empowering people through play” in the morning session, and I have to say it has been one of the best professional decisions I have made attending this workshop.

Christina and Nicola are great proponents of the power of play to engage and develop people within the workplace. They have worked together at The App Business, and have developed a great rapport with both each other and the folks in the workshop.

Through a series of activities, both practical and thought provoking the group were encouraged to develop our thinking and learning to enable us to solve problems and adapt to change. 

The activities were: 

  1. In a circle, we each gave our name and revealed a fact about ourselves. We then went round the circle again and had to recall the name and fact of each person in sequence. This was as much a challenge as it was an icebreaker. Memory is hugely at play here but it does fail you sometimes. Some names and facts were easier to remember tha others, perhaps because the were unusual. The rhythm and pattern of the sequence almost became second nature by the end of the game, so by the end we had all gotten to know each other through the confines of the activity. 
  2. Lego story boards  – using an iterative process, we built a narrative story board, an then constructed the narrative in Lego. At various points in the activity, a new element was introduced, or a complication that meant we needed to replan and refactor our work. In short order, we discovered that our resources, imagine and ability to adapt was to be put to the test.
  3. Project Jenga – the team were split into three groups, developers, testers and designers. With Nicola acting as a project manager, we were asked to design and build a mobile application to accompany the conference. Through discussion, we had to meet certain acceptance criteria, explore problems and risks. With each encounter we had to remove a block from a Jenga tower. With each move the risk increased that the project (or Jenga) would collapse. This activity allowed us to explore our questioning skills, as well as our empathy and co-operation with other teams.
  4. Posters – in groups we were asked to design a poster for TestBash, using certain acceptance criteria. We then wrote a description of our poster which was then shared with the other team, who then had to draw a poster using our instructions, and vice versa. A fantastic activity that dug into our ability to analyse and interpret instructions and acceptance criteria, whilst engaging with the others in the team in a visual medium. We then compared each teams efforts to see which was closest to the desired product.

   
  

   
Nicola and Christina did a fantastic job. I said on Twitter that this was one of the most fantastic learning experiences I have ever had, and I meant it. I still have a lot to analyse and interpret, so that I can apply the learning practically in the workplace. Maybe we can use some of these activities or others to enhance our communication, our empathy and our ability to adapt.

TestBash Brighton 2016 – Conference

The conference day in hindsight was a bit of a blur for me. With all of my might I tried to concentrate on Lisa Crispin and Emma Armstrong‘s opening talk “Building the Right Thing: How Testers Can Help”.  Unfortunately (or fortunately) I was up next on the bill, so I was a little distrcted trying to maintain focus on not collapsing in a heap. Lisa and Emma kicked off TestBash with a bang, with an insightful exploration on how testers can be the guiding light on projects, ensuring that not only teams do the job, but do right. 

Lisa Crispin and Emma Armstrong

Lisa Crispin and Emma Armstrong

After the break, one of my favourite talks was Katrina Clokie‘s “A Pairing Experiment”.  This talk described and explored how Katrina lead an developed pairing activities within her team at the Bank of New Zealand. When questioned on the challenge of convincing managers to relinquish team members for paired work, she responded that whilst they might be losing one tester for an hour a week, they’d be getting an extra tester for another hour each week. I’ve been lucky enough to see Katrina speak before about her work, and each time it’s been a revelation. She works hard to develop the testing at every organisation she has worked at, an beyond into the community itself. Great stuff!

Katrina Clokie

Katrina Clokie

Up next was John Stevenson (@steveo1967). In his half talk/half conversation “Model fatigue and how to break it” he invited us to examine critically the models we use for testing every day. 

He challenged us to reevaluate the models we use, cut them up, adapt the ones we find successful, combine them with other models, throw away ones we don’t use, create new ones if needs be. He challenges us to be diverse in our approaches to testing, no relying on this are models all the time. That way we can find out more interesting information about our testing. John is a great presenter, who engaged and enthused the audience, inviting them to through questions at him for he last 10 mins of his time on stage, rather than talk to the end of his time.

It’s something I can reflect on in my own day to day work, where we use a model to evaluate and plan our user stories the testing that is discussed in those stories. It has been adapted and changed over time to suit our needs, and I am sure will be changed or even chucked away if it doesn’t suit our purpose in the future.

 

John Stevenson

John Stevenson

 
Later on in the day we heard from Patrick Prill, in his debut talk ” Accepting Ignorance – The Force of a Good Tester”. He led us in a discussion of how ignorance is not necessarily a wilful lack of knowledge, but just an absence of knowledge. That developing through understanding where our ignorance exists, we can develop our knowledge. It’s a huge force for change in testing. This is the gap between what we know, and what we don’t. The reflection on that upon our work as testers is where this talk had its greatest impact. 

Where Patrick was not an experienced speaker (you wouldn’t know that from his talk), he utilises his many years of testing experience in Germany, the problems both cultural and technical that he encounters in his work which gave his talk huge insight. 

Patrick Prill

Patrick Prill

After lunch we had our guest speaker, Grammy award winning singer (and tester) Michael Wansley (@teewanz) give us a highly entertaining, somewhat controversial but engaging talk “Test/QA A gatekeepers experience”. Testers as gatekeepers is not a very popular paradigm amongst the vocal members of the (particularly) context driven testing community. But within the wider view of testing as a process that is involved in developing and selling products, gatekeepers are often what testers and testing are perceived as. 

It’s a popular view (one that I subscribe to) that testers should be information providers, learners, investigators but not necessarily decision makers about whether software ‘goes live’ or not. We may be part of that decision making prcoesses, but not the arbiter of it.

It’s within this context that I have a certain amount of empathy with Michael’s experience of working on a number of iterations of the Microsoft Windows operating system. He understands that testing cannot exist in a vacuum, where there isn’t recourse to customers, managers and Vice Presidents, or consequences of screwing up. His talk did (quite rightly) invite comment, and Michael stood up for his view honestly and with vigour. Whether you agree or disagree with his view, that should be applauded.

 

Michael Wansley

Michael Wansley

  
Zachary Borelli introducing Michael Wansley

Zachary Borelli introducing Michael Wansley

 

After that it was “Having all your testers code: It doesn’t have to be a big deal” by Anna Baik (@TesterAB) and Andrew Morton (@TestingChef) on the challenging task of ensuring all testers contribute to the automation strategy at Brightpearl. Now I have to give some personal interest here, as not only are Anna and Andrew friends of mine, they are also former colleagues of mine from my time contracting at Brightpearl in Bristol.

It’s a fast paced, highly charged environment of great development and testing across the business. I was tasked with testing integrations between the Brighpearl service and a number of third parties. I didn’t get too involved on the automation side of things, but I do know what a challenge it was to implement. 

This was I think for many a challenging talk to follow, as the style was unusual (no slides), but the content was highly pertinent and valuable to many teams now trying to grow and mature their testing capabilities and automation strategies.

Anna Baik and Andrew Morton

Anna Baik and Andrew Morton

As a tester who too often focuses on the technical rather than human elements of testing, the next talk turned out to be my absolute favourite of the day. “Do testers need a thick skin, or should we admit we’re simply human” by Nicola Sedgwick (@nicolasedgwick) was a bold and brave exploration our ability to communicate, or failings as testers to sometimes not recognise problems not with software but in ourselves.

One of the key aspects of this talk was our response to stress, how it compounds upon other stress. Where there is a lack of challenging activity, or work we care about can lead to either boredom or even more stress. Some of my close friends in testing know that the last couple of years have been difficult for me, professionally and personally, and for this reason this talk really resonated with me. Nicola challenged us to ask what kind of tester we were. Well, I’m not sure I can answer that question yet, but I’ll be one that never forgets that humans are fallible, in a world that increasingly looks to punish those who fail to realise that.

 

Nicola Sedgwick

Nicola Sedgwick

 
So to the final talks of the day – next was my friend and mentor Bill Matthews (@bill_matthews) who introduced us to the concept of Smart Algorithms. The maths and logical flows that allow systems to learn, recognise patterns and process data based on a wide range of inputs and variables. He challenged us to examine the potential testing concerns that might arise from working within such applications – a really complex problem which Bill was able to present with humour and deep, practical knowledge. I have to add here, that with glasses I am a Golden Retriever, but without I am a German Shepherd.

Bill Matthews

Bill Matthews

And finally…Nicola Owen (@NicolaO55) also from New Zealand, but recently relocated to Sweden to work with the great folks at House of Test. In “Nowhere to hide: Adjusting to being a team’s sole tester” Nicola guided us through two case studies were she was the sole tester on two very differs projects. She reflected upon her experience with great depth, clarity and insight, what she learned, her developing confidence and skill. In one case study she felt insulated from the problems that software development teams encountered, and in the other far more exposed as the sole tester. In each she presented how she approached each problem and dealt with it head on. Another awesome talk, to round off the day.

 

Nicola Owen

Nicola Owen

 
So to round off the proceedings, our host Vernon Richards (@TesterFromLeic) and his able assistant Mark Tomlison (Mark Tomlinson) lead us into a round of always amazing 99 second talks. This is the first time I have not done a 99 second talk, so it was refreshing to just sit back and enjoy. Highlights for me were Emma Keavney’s rap (@EmJayKay80) and Deborah Lee’s sit in (@DeborahLee89). Also a special mention to the new Software Testing Clinic (@TesterClinic) announced by Mark Winteringham (@2bittester) and Dan Ashby (@danashby04), which I hope to get involved in soon! Well done to all involved. A great potential showcase for future speaking talent I hope.

 

Mark Winteringham and Dan Ashby from Software Testing Clinic

Mark Winteringham and Dan Ashby from Software Testing Clinic

  
Deborah Lee

Deborah Lee

  
Emma Keavney

Emma Keavney

 
So, to wrap up, TestBash 2016 I felt was an enormous success, both from a personal point of view, and in terms of the rude health of the conference. Rosie has done a great job again this year, and I hope to be involved again in future.

 

After party with Jess, Rosie and Helena

After party with Jess, Rosie and Helena

  
Breakfast with Chris, Nicola and Martin

Breakfast with Chris, Nicola and Martin

  

It’s all about the conversations – TestBash 2015 Review

Firstly, a preemptive strike for my love of TestBash.

I make no bones about it, I love this conference. No other expression of emotion comes close. Its almost up there with my wife, family, friends, my cat and Doctor Who. (And to anyone that knows me, that is a pretty big deal)

Regardless of the quality of the conference track, speakers and workshops, this annual event is now rapidly becoming a part of me, my learning as a tester and driving my desire to evolve my testing. It also helps me support and mentor other testers – both those I work with, and those I don’t.

As I mentioned in my previous post, where I previewed TestBash 2015, if it hadn’t been for TestBash I most likely wouldn’t be working where I do today, with a company I enjoy working for, and a team that I admire and value. I also wouldn’t have had the courage to do any public speaking or workshops if I hadn’t attended TestBash in 2013. As long as it is running, and as long as I can attend, I will go. With some luck and preparation, I hope to be more involved in TestBash 2016!

Now with the context of this blog post set out, I’ll try to present my ‘impartial’ review of this conference. It’ll be hard!

For the last three years I have made a pilgrimage back to my home town of Brighton to attend TestBash. Each year it has produced a different mix of learning, excitement, comradeship and an emotional exhaustion that my friend and BrighTest organiser Kim Knup has aptly described as the post TestBash blues. Through TestBash, social media acquaintances have become colleagues in testing, and in some cases firm friends. I may only see them for a few hours a year, but for that, above anything else I am grateful to Rosie Sherry, Simon Knight and all of the Ministry of Testing team that run the event.

Brighton Pavillion at Night

Brighton Pavillion at night

I took the photo above of Brighton Pavillion, whilst having a fantastic chat with Stephen Janaway on our way to the meetup on the Thursday night. And it is this that indicates the value to me of TestBash as a whole. It’s all about the conversations. Stephen was not the first great chat that weekend, nor was it the last. We discussed testing, my poor recollection of the geography of Brighton seafront, our upcoming conference talks and workshops and even family. I suppose you could say that the testing community, formed around this conference has become as sort of family to me.

Here we are at dinner with Chris Chant, Vernon Richards and Rosie. For me, the conversations start with the small events and gestures like this, and reminds me that I owe Rosie dinner! It had become a bit of an in joke that Vernon was going to wear a tutu on stage on the conference day, and in the end he did, but not in the way you might expect.  More on that later. I was lucky enough to hangout with some of the conference speakers and workshop facilitators at dinner, discussing their experiences and feedback on the day. As conferences and workshops go, it very good value for money, as the Ministry of Testing is able to attract some high calibre speakers and contributors every year from across the community, even just to attend!

Chris, Vernon and Rosie at Dinner

Chris, Vernon and Rosie at dinner

Sadly, I was unable to attend the workshop day on the Thursday. However, I was able to catch up with some folks at the end of the day down at the Brighton Dome. There was an open meetup and test gaming session to wrap things up, so I watched a round of Set, and led a few testers in a few rounds of Zendo. If it hadn’t been for a lunchtime learning session with my colleague and friend Chris Simms a few months ago, I wouldn’t have had a set of rules in my head ready to play! All power to the test community. Even though he hadn’t attended this year, Chris’s impact was felt from afar!

Ryan and Danny at the Meetup

Ryan and Danny at the meetup

So, off to the meetup, at a bar I hadn’t been too since my early 20’s. We took a minor detour on the way, but got there in the end. Here is my colleague and good mate Danny Dainton, enjoying a drink with Ryan Rapaport, a representative of one of the conference sponsors QA Symphony. (Shameless Plug 1: I use their tool QSnap, it’s pretty good).

The greatest value of TestBash for me comes from the conversations had at meetups like this. Sure, there was a lot of talk about testing, about our experiences of testing, our learning from various books and speakers, the relative merits of one conference over another, the relative merits of one beer over another. Here I was able to catch up with my (Shameless Plug 2) Weekend Testing Europe colleagues Neil Studd and Amy Philips, and plan our ground breaking trio 99 second talk for the following day! I also managed to grab conversations with; Matt Archer, about the Ministry of Testing Dojo and Abbie Maddison, the new runner of the NottsTest meetup. It was also fantastic to catch up with Guna Petrova from Latvia, who is a key player and track organiser at Nordic Testing Days. Her outlook on testing is always refreshing and enlightening.

Without communities like TestBash, and those generated around other conferences like Let’s Test, Weekend Testing wouldn’t exist. Communities generate conversation, which lead to initiatives and plans, which lead to more communities and more conversations and deeper learning experiences. Similarly, though meetups like this, there are opportunities to develop professional relationships, which can lead to other meetups, brown bag sessions, invites to speak at conferences, or even work!

Weekend Testing Europe: Amy, Neil and Me

Weekend Testing Europe: Amy, Neil and Me

Later in the evening led to even more discovery and exploration of our craft (testing, beer and music). It with great surprise that I could discuss the merits of the music of Fairport Convention and Jefferson Airplane (whom, thanks to my Father, I have an appreciation of) with Michael Bolton and Neil Thompson.

But that isn’t really what we were there for. Here’s Radomir Sebek, a tester from Serbia, who works for a music production software house in Berlin. He’s playing “The Pen Game” with Michael, one of the many testing games that were going down at The Globe late into the night. That same conversation led me to be challenged on a variation of the Pen Game, this time with my observation and listening skills put to the test. I got the solution, in the end!

The Pen Game with Michael and Radomir

The Pen Game with Michael and Radomir

Richard challenges Abby and Dan

Richard challenges Abby and Dan

Above is conference speaker Richard Bradshaw challenging Abby Bangser, from Thoughtworks, and Dan Caseley, from Common Time, to more testing games over a beer or three.

So here is the problem. With so many fantastic folk to talk to and learn from, you can’t really chose from them all. You pick up on different sounds and movements, explore what is interesting to you, find people you have never met before, or have had online communication with. It’s a bit like (exploratory) testing, in that you can define your conference by the actions you take, the information you gather, the people you speak to and your responses to them, and how you record them…like this.

So to the main event.

Each year, Rosie manages to attract excellent speakers to TestBash. And this year was no exception. As I mentioned in my previous post, there was no diversity in terms of gender at the 2014 conference. Not so this year, with three female speakers on the conference track. I have no details on the selection process, but I feel that the overall content, tone and message of the conference was all the better for the selections made this year.

There was also a lot to learn, from a range of experience reports, new thought leadership and science around testing, as well as technical challenges. Where TestBash is usually strong is dealing with the human element of testing, rather than drowning the attendees with technical jargon. Testing is for me very much a social discipline, as much as it is a technical discipline.

First up was Michael Bolton with “The Rapid Software Testing Guide to What you meant to say”, which looked to our use of language as a tool of our trade, and challenged many potential assumptions that could be drawn from testing behaviours. It’s my interpretation of this talk that Michael was trying to draw out the reasoning behind certain language choices in software development, and in some ways subverting their use through the prism of context driven testing. Why for example would we say automate all the testing, where we couldn’t possibly do that with development?

Up next was Iain McCowatt, with an excellent and animated discussion of the need to include intuition and the importance of tacit knowledge in our detection of bugs.  Iain emphasised that socialisation and interactional expertise was an essential skill of testing.  Being able to discuss and share our work and experiences appear to be key in finding bugs and communicating them effectively. It was also a great reminder to pick up the work of Harry Collins, whose writing and research contributed greatly to the themes Iain was conveying. I managed to catch up with Iain during a break, and sought his advice on combating biases in my testing. I find sometimes that because I test a lot for security, I feel that this sometimes blinds me to other considerations whilst I am testing. His insight will be invaluable in trying to balance my approach and test design processes in future.

Next up was an interesting talk about the challenges and learning gained from The Guardian’s approach to mobile testing and delivering software across multiple platforms. Sally Goble and Jonathan Hare-Winton presented a fascinating and humorous exploration of the differences and pitfalls of testing on both the iOS and Android operating systems and associated hardware. Playing on the rivalry in historic advertising campaigns between PC and Mac, and a distinctly divided audience (seemed to be more Android users than iOS, but only marginally so). This was a great talk for me, as I know very little at all about mobile application testing. The style of presentation drew more out of the audience than I expected it would, and it did not dwell too much on technical details. Great stuff!

After the break came the double bill of Martin Hynie and Stephen Janaway. Both talks approached the problem of organisational change and perceptions of testing and test management within development teams and businesses as a whole. Placing these two talks together was a masterstroke, as they complimented each other so well. Martin’s talk “What’s in a name? Experimenting with Testing Job Titles” focused on a social and professional science experiment. Martin found that following a change in job title and team name, to remote test, or testing; enabled his teams to have greater impact and authority within the business. He did all this under the radar, with the testers maintaining their responsibilities, whilst having a different job title. With an exciting presentation style, Martin was able to convey that maybe businesses see testing and testers as limiting and a blocker to progress. In doing so, he discovered that other teams and key stakeholders responded more positively to the alternatives. There is a lot to discover in this talk, and I won’t spoilt it further for anyone who want’s to watch the video when it comes online. Let’s just say for me that Martin’s talk it is one of the highlights of the conference.

To Stephen’s talk. For a while now, Stephen has been an inspiring member of the testing community, both personally and professionally. I was invited to speak to his team at Net-A-Porter last year, which was a fantastic opportunity. So its exciting to see how he managed to evolve into his new role as a Testing Coach, in his talk “Why I lost my job as a Test Manager and what I learned as a result”.

Organisational change is a very real challenge for testers. Stephen’s experiences here are both common, in terms of the need of testers to adapt professionally to change, but also uncommon in the approach taken by Stephen’s organisation. Rather than having overlapping development and test managers supervising the work of many people across teams, each team had its own development manager.

As a testing coach across the whole business, Stephen’s new role is to mentor the testers, enable and guide their professional development and learning, whilst not being responsible for their line management. This must have been an awesome task, reorganising the development team of a major online retailer, whilst at the same time maintaining delivery of products and services. This was an experience report beyond the normal recollection of events and dry facts, and really drove home that testers need to be able to be at the forefront of change in organisations, rather than being reactive to it.

Vernon Richards was up next, with “Myths and legends of software testing”. In 2014 Vernon blew the house down with his 99 second talk on this topic; a rapid fire list of misconceptions, musings, biases, and warnings. What Vernon did here was to distill the core of his message into an blisteringly and entertaining talk. After lunch and with everyone feeling a little full, it was the best of antidotes to wake us up.

Vernon’s talk drove home the need for testers to not only be creative in their approaches to testing, but to be wary of the fallacies and biases that can be derived from poor research, assumptions and inaccuracies. Also, looking at how to challenge the language used to describe testers and testing by non testers; such as “It’s just clicking a load of buttons” or “Anyone can do testing”. If we are to take ownership and responsibility for our craft we have to believe in our skills, and champion them to those outside testing, so that they are recognised and valued appropriately.

Maaret Pyhäjärvi came next, with “Quality doesn’t belong to the tester”. Maaret’s experiences of being the sole tester on the team, feeling responsible for quality when it seemed that no one else appeared to care resonated with me deeply. This story described how she managed approaches to testing on her team and began to build more positive relationships with the developers. In order to test sooner, and test better, Maaret elicited a collective responsibility for quality and testing, rather than taking on the burden on her own.

Matthew Heusser encouraged us to rethink our approach to regression and releases in his talk “Getting Rid of Release Testing”. This talk lead us through an approach to testing and releasing software incrementally, and becoming less reliant on the big bang “test everything” approach to release management.

Through drawing rather than slides, Matthew explained what he termed “The Swiss cheese model of risk”, where at each stage in a software release life cycle there can be different layers of testing, where there will be gaps and overlaps in coverage. It’s probably a scary approach for some, but resonates with me as working in a continuous delivery environment means that to test everything at the end would be inefficient, costly in terms of time and resources and likely not give us meaningful data. The tweet below reiterates clearly one of Matt’s main messages in a challenging and insightful talk.

Nearing the end of the main conference day leads us to Richard Bradshaw’s “Automation in testing”. I’ve never seen Richard speak before, but I have heard much about his ability to convey complex thinking in a clear and approachable way. I was not to be disappointed. Richard guided us through his evolving process of  supporting testing using automation. Built up over a number of years of learning and experimentation, he described a mature and adaptable way of incorporating automation into your testing, for the right reasons – enabling the important checks that you might need to do frequently, allowing the tester to focus on exploration, learning and asking questions about the software under test. This was an inspired and entertaining talk, which engaged me in a topic that in the past has not always held my interest.

Now to the final presentation of the day, with Karen Johnson’s “The Art of asking questions”. This was hands down my favourite talk of the day. It was less of a presentation, more of a conversation with the audience. Karen’s slides were a simple guidance to invite us to flow through the discussion with her.

Karen explored with us the finer points of questioning, both of others and ourselves. Timing was a key theme, asking the right question at the right time, something I have struggled with in the past. Even more resonate with me was the idea that, quoting author Joshua Harris “The right thing at the wrong time is the wrong thing” in his book I Kissed Dating Goodbye: A New Attitude Toward Relationships and Romance

Drawing on her journalism background, Karen asked us to consider the kinds of questions we ask and how they might influence the kinds of responses we get in return. The classic, yet always useful what, where, why, who and how that will never fail you as long as you use them appropriately. After all, a lot of testing is about asking questions, and asking the right question could even prevent defects from occurring before a single line of code is written. The Q&A afterward brought many excellent questions from the audience, with Karen responding with great advice, book recommendations (see Twitter for a tonne of them) and practical suggestions to solving communication issues.

TestBash has now established a tradition of 99 second talks, led for the final time by Simon Knight. Many great folk stepped up to the stage alongside Neil, Amy and myself. Jokin Aspiazu really coined it with “If you can’t get money for conferences, ask for time. Time is valuable.” No truer thing has been said in such a short space of time!

The after party is both a chance to relax after a long day, but to engage with as many people as possible. The quite excellent and intimate bar The Mesmerist proved to be a great place to hang out and talk testing, such as with Mark Tomlinson (he of the infamous spinning cat at TestBash 2014).

Mark Tominson at the meetup

Mark Tomlinson at the meetup

It’s the camaraderie and convivial atmosphere that really makes this event, year in year out. I recommend you come, make a week of it…to really let Brighton and TestBash soak in to you. You won’t regret it.

Reflections in a single malt

Reflections in a single malt

Although, I might do by the end of the evening

A community in contrast

I’ve not blogged recently for various reasons, both personal and professional. But on the anniversary of my blog, I want to return with a more positive attitude to it after a fallow period. This is a quick blog as way of a catch up over the last few months activities (other than my professional and personal ones). It’s an opportunity to share some of the highlights of my experiences in the testing community recently, which have been warm and welcoming during some difficult times.

A few months ago I attended the inaugural Brighton Testing Meetup, catching up with some of the good folk I last met at TestBash 3. Brighton is sort of my home town, yet I have never worked there so having a foot in the pond that is the testing community there has been a great thing. We talked, we ate and drank and shared ideas. Early plans have been made for my future involvement, leading talks and discussions around some exciting testing topics. Emma Keaveny and Kim Knup are developing a vibrant new community of interest and I can’t wait to be more involved. Roll on 2015.

The community of testing is as varied and as exciting as the variety of people who work and learn within it. This is a good thing, perhaps the greatest thing about the community…and this is where the contrast lies.

The same week I went to Brighton, I also attended the latest Special Interest Group in Software Testing conference. SIGIST is organised by established, more academic people in the testing industry, on behalf of the BCS. It meets quarterly in London. There were a number of interesting topics being discussed, but it didn’t set my heart on fire. Only one or two talks out of the whole day really engaged me with the subject matter. Whilst there was the opportunity to learn from some experienced practitioners, there  wasn’t the same emphasis on collaborative learning, challenging established testing paradigms and positive enquiry. It wasn’t a bad experience, it just didn’t make me more passionate about my craft, nor help me understand something new about testing. It was good however to catch up with some people who I have met before, and some who I hadn’t…but were on my radar. Namely Tonnvane WiswellDeclan O’Riordan, Paul Gerrard, and Mike Jarred.

Another recent experience has been with some of the free, online and collaborative forums for learning and discussion that I have participated in. Firstly, Stephen Blower’s Testing Couch forum. This is a free and open Skype forum for any testers who are interested in talking about their craft. In the couple of times that I have attended, the chat has always been productive, supportive and non judgemental. Stephen makes this forum available periodically, usually every month or two. It’s a fantastic opportunity for experienced or novice testers to throw ideas around, be challenged and share thinking and learning.

Lastly, and probably my most positive experience was being a guest speaker in October’s Weekend Testing Europe forum. I was sharing my recent learning and experience in software testing, leading the attendees in an exploratory session with security as the focus. To a lot of the people during the chat, security testing was a new concept for which they had little experience or opportunities to learn. It was incredibly rewarding to be able to facilitate this session, not only on a personal level, but also to see many others taking up the challenge of securing their applications, and considering security as part of their testing.

Amy Phillips and Neil Studd have really breathed new life into Weekend Testing Europe, which had been dormant for a while. Keep an eye out for WTEU in the future, as it is a great way of keeping in touch with the testing community around the world. Be prepared to go in with eyes open, lots of questions, and a hunger to learn. All you need to do is  volunteer two hours of your time on a Sunday afternoon. It sure beats watching Columbo repeats or traipsing round a garden centre.

So, that’s it for now. I’ll be blogging again soon. The Test Doctor will return!

Engineering the solution together!

Like a lot of teams, my team is cross functional. There are several engineers with different specialisms…systems development, performance and security, as well as testing like myself.

When we find an issues, we work on them together as a team. When we test code, we test it together as a team. We celebrate our successes and solve our problems, together, as a team.

The other day I uncovered an interesting bug. It was the sort of bug that still gave me that little buzz of excitement that I got when finding bugs earlier in my career. Not only was it a challenge to identify, replicate and describe but because it was found in a new piece of development which will really help our team solve an important problem for our customers and our service.

If as a team we can work together to solve this problem, then this benefits the customer and ultimately the business. It’s all we are about, day in day out.

The developer who is working to fix this issue feels that has identified the root cause, which has taken a lot of time, effort and frustration on his part. Research, experimentation, consulting with myself and other developers…wash, rinse, repeat…until the solution is found.

Tomorrow morning we start testing (in a VM on the developer’s machine) a pretty radical fix to the bug, which will mean a lot of testing effort for us both before we let it loose in the wild.

To be honest I can’t wait to get to grips with the solution to this problem. The testing challenge will be one of the toughest since I have been working within this team. I’m not sure yet what it will exactly entail, but the fun will be in discovering that in the days to come.

Tales from Tallinn

What is it about testing that I enjoy? Is it learning new skills? Certainly. Is it getting to work with really talented people? Of course. Is it sharing ideas and thoughts with the community? Most Definitely.

You get all of these in spades at Nordic Testing Days. Great tracks with a wide variety of speakers, workshops and keynotes. Contributors from the USA, UK and across Europe make this a very diverse and exciting conference all within the beautiful city of Tallinn.

Tallinn is a fantastic city with lots to offer. The medieval old town really knocked me off my feet, with its street markets, vibrant buildings and massive range of pubs and restaurants. The city really comes to life at night, as the sun had barely set even at midnight. The speakers dinner was spectacular also.

20140609-220552-79552224.jpg

The organisers run a tight ship, ensuring that speakers don’t over run and resources, food and drink are always available. Here are two of them; Helena Jeret-Mäe, who is a member of the content team and manages all the marketing and PR and Grete Napits, who is the conference chair. They did a fantastic job as our hosts and organisers.

20140609-205814-75494992.jpg

I was lucky enough to travel over with two of my colleagues; Rob Lambert and Raji Bhamidipati.

Raji facilitated a great workshop on the fine art of note taking, a skill that we as testers need to hone in order to be effective at our jobs. She provided interesting testing challenges for us, but the biggest challenge was being able to communicate our testing effectively using a variety of note techniques. It’s difficult not being biased here, as Raji is my friend, but she worked damn hard making sure that workshop was good…and succeeded.

Here she is in action:

20140609-214554-78354512.jpg

For me though, the stand out talks came from Stephen Janaway and Gitte Ottosen.

Stephen talked about how our emotions can be both a blessing and a curse when testing, both in terms of how we communicate effectively with other team members but also how emotional biases impact our thinking and judgements. It particularly struck a cord with me as in the past I will freely admit that I have let my emotions cloud my judgements in the workplace.

20140609-213619-77779022.jpg

Gitte was speaking about how being pragmatic rather than dogmatic about particular test processes and techniques can lead to getting the job done successfully and professionally. Gitte has spent a lot of time working with the Danish military, both as a soldier in the Danish Air Force but also as a tester and manager on IT projects. Her experience and learning through these processes was presented in a clear and succinct way, that didn’t pander to popular schools of thought on how test projects should be run.

20140609-214501-78301630.jpg

The closing keynote on the first night, and we had the great pleasure of being double teamed by Matt Heusser and Pete Walen. They presented a live testing challenge to a room full of testers. Whilst a deceptively simple problem, everybody had different ideas and thoughts on how to tackle it. A really exciting, and practical keynote.

Unfortunately due to having to fly home on Friday afternoon, I missed Pete’s talk on leadership…influenced by the world of Harry Potter…which intrigued me as I am a huge fan of The Boy Who Lived and his adventures. Pete and I managed to spend some time together prior to this, letting me get an insight into his talk.

I also missed the closing keynote by Iris Classon which I heard great things about. I’m looking forward to catching the videos as they come up on the conference website.

I was speaking about security testing again, giving what might be my last run at ‘New Adventures in Software Testing’. I had some great feedback on the day, and since, which I am most grateful for. I hope to come back to the circuit next year, with some new ideas and thoughts on how to deliver an effective security testing strategy.

Virgin Territory

Image

 

By the time you read this post the above image will be out of date. Romania Testing Conference is held in Cluj-Napoca in the north of Romania. You can find out more about it here. It is an intimate multi-track conference followed by a workshop day. There were a lot of local speakers, plus some from Holland, India, Spain and the UK. As I arrived late on the first day, I was only able to see a few speakers on the first day. 

Firstly Luis Faile, who talked about “Myths of Exploratory Testing”. A lot of his talk resonated with me, as often when I have talked to other testers or IT folk who haven’t experienced Exploratory Testing before, these are the questions that they generally want to ask…such as how do we measure it, who is accountable, what evidence is there etc etc. Luis deals with these issues in a pragmatic way and invited experiences and suggestions from the floor. Altogether a great way for good testers to champion and communicate about this particular aspect of testing to those who are not aware of its benefits.

Last up for me was the opportunity to sit on on Andy Glover’s (http://cartoontester.blogspot.co.uk/) Visual Testing workshop. Here he encouraged our creative abilities by getting us to think about how we can communicate visually. He inspired us to explore how visual aids such as diagrams, annotations, cartoons etc can be used to describe information, data, bugs, ideas and other testing concepts. Andy was inspired by Dan Roam’s book The Back of the Napkin which is definitely one to add to the reading list. 

Image

RTC 2014 was a great experience for me for a number of reasons. Not only was it my first ever international conference, it was also my first opportunity to speak at a conference, running a workshop on security testing. If you are interested my slide deck is available here

The testing community in Romania is vibrant and rapidly growing. It was a great chance to see how other testers from outside the UK learn and develop their skills. All the testers that I met and talked to, either from Cluj, Bucharest or other parts of Europe, were enthusiastic about their craft, keen to learn from each other and the speakers on the circuit. This is rewarding to see, as the key objective from my workshop was that other testers would take back what they learned to their places of work and where possible implement that learning, and ultimately add value.

Image

I was grateful to be invited to speak there and would relish the chance to do so again. I’ll be speaking at Nordic Testing Days in Estonia in a few weeks time, so expect a post on that event soon!

Reinventing Regression

Manual regression testing has always been a burden for testers. It’s one of the practical problems that we face, where there is a gap at the top of your testing pyramid that can’t be checked using automation. At NVM we have a suite of automated tests which do a lot of the leg work with regard to checks. This post isn’t going to be a discussion about the difference between testing and checking, or the merits of automation over manual checking. It is simply a demonstration of a problem we faced as a team, and a solution we came up with as a team.

We realised that we had a problem. Some time ago we found that manual regression was only being done by two or three people and it was taking hours and hours of those individuals time. And it was usually the same two or three people for each release. It was taking those individuals away from their ‘more exciting’ feature teams work, where we do a lot of funky exploratory testing. But more than that, it was creating division and ill feeling amongst the team where some individuals felt they were carrying the burden.

Also we saw that we weren’t getting the visibility of the process that we wanted, such as the number of checks we were doing, the quality and appropriateness of the scenarios under test; or whether these checks were already (or could be) covered by automation. We didn’t want to duplicate effort. All of these tests are maintained on our internal wiki, rather than in some impenetrable test tool. We are all responsible for maintaining the regression suite, so if we feel something needs adding or changing, we take initiative and do it ourselves.

The scenarios went through a process of review and streamlining over a number of iterations, to make sure that we had the most best possible set of checks we could. These were all described in terms of agents and supervisors operating within a contact centre, which is of course a core part of New Voice Media’s business.

Getting a kit ready for live deployment is the top priority for us, and we want to release as regularly as possible…weekly if resources and time allow. There is of course a challenge to manage the needs of our own feature teams and their priorities, but of course there is a business to support, so the release takes priority.

So to the physical execution of the regression…a whole challenge in itself.

At New Voice Media we are lucky enough to have a dedicated test lab, where we have desks, networking and telephone handsets that allow us to run our tests in one place. All available testers meet at a set time, with our PC’s, telephone handsets, tablets and other tools.

We have created a Kanban board, along with some other elements. We have story cards for each of the scenarios under test, with two separate groups in the team handling different logical streams of the application. The board allows us to see progress through the testing, delegate tasks but also gives us a chance to visually provide instant feedback to what worked well and give praise, what didn’t work well and any ideas we might have to improve the process.

This is a great example of skilled testers working together to solve a testing problem, and has started to make regression testing an enjoyable event rather than an onerous chore.

Image

 

The journey starts here…

Welcome to my new blog…



I’ve been testing for a while now. Some 12 years. Only in recent years have I felt the need to give something back to the testing community that for a while I felt disconnected from…and now is the time to make that desire a reality.

Anyone who knows me will understand why I have chosen the moniker The Test Doctor (@thetestdoctor on Twitter), but rather than filling your feeds with useless geekery, this blog and any posts from this twitter feed etc. will be entirely testing, IT and other related topics. 

I am currently a contract software tester (@drbconsulting) and my current client is a firm in Basingstoke, Hampshire in the UK. I work with a lot of great people, with a huge amount of knowledge and experience to share. It’s fast paced and growing, with a huge challenge for developers and testers alike. But because the team work so well, those challenges can be shared and reflected on in a very positive environment…something that was lacking earlier in my career.

In the last six months I have learnt more about software development, the testing challenges that face us as a community, and myself as a tester than I have in the last ten years. Too much to include in this initial post, but over time I hope to be able to share what I have learnt with the community.

I hope to write periodically over the next few weeks and months as time allows. Like all of us, I have a lot of family and personal commitments and outside interests. So anything I post will be largely dependent upon those factors getting priority.

Firstly a little bit about me as a person. 

I currently live in Frome, Somerset, not far from the city of Bath. I am originally from Burgess Hill, near Brighton in East Sussex. I have been married to my wife Rae for 8 years.

I run a Cub Scout pack in Frome, Somerset, and also I am Assistant District Commissioner for Cubs for the Frome, Wells and Shepton Mallet area. It’s a huge mental, emotional and physical challenge; but it teaches me new things about being a human being in a community of other humans every day. I have to work with adult leaders, other volunteers, parents and children to make sure they have great adventures.

I studied as a Primary School Teacher at Bath Spa University (yes, Bath has an ‘other place’ too) but during my final year decided that this particular vocation was not for me.

I started my IT career whilst working at the AOL/CompuServe call centre that was then in Bristol. Within the year I found myself testing screen pop and cased based reasoning tools whilst handling calls, and eventually a full time promotion to the Tools team, where I started on localising and testing a new CRM tool for the business.

Following this initial foray into testing, I worked for various companies such as Capita, Yell.com and Northgate Public services. For anyone who is interested, my Linkedin profile can be found here: http://www.linkedin.com/in/danielbilling

I started contracting in 2010. The dual motivation of choosing my own clients and developing my own future challenges appealed to me. But partly because I felt somewhat adrift in my career, with what I perceived as a lack of focus and perhaps commitment. It had become a  means to an end, rather than a career in which I was interested in and wanted to develop.

In the last couple of years I have found a new impetus to learn more, develop myself as a tester, learn from other people in the software industry and the testing community as a whole. I may have been testing a while, but for a long time I had blinkers on…just getting the job done, looking for that next rung on the ladder so I could improve my lot. 

Learning new tools, techniques and processes is all very well. But unless you have some sort of personal framework or context in which these learnings can be utilised, they just become bullet points on a CV. Meaningless! 

Unless you can prove yourself as a competent, learning and developing member of the testing community, these skills are not worth the paper they are printed on. Sure, I have taken those courses we all debate about, and worry about their validity and use, but to be honest I don’t want to work for organisations where having letters after your name are a prerequisite to getting a job. I want to discuss things with my peers, develop great ideas, materials and resources to share with the community. 

In the last year I have been given a metaphorical and physical kick up the back side by attending just one or two events with my peers. Alongside that, I have worked within a couple of organisations recently where you really needed to raise my game in order to stay ahead of the testing challenge. 

I could reel off a list of influential people who have helped me back onto a new path where I feel happy to contribute to a vibrant and exciting community. But that would be name dropping. Those people I hope know what they mean to me. To them I say thank you. To the rest of you, I hope to meet you soon, learn from you, maybe help you learn something too.

I’ve got some great ideas in my head. I want to get them out there to share with you. If they are flawed, I want to know. If they need challenging, then throw them back at me. I look forward to meeting you!