Learning from the other side: Thoughts on conferences, workshops, learning and test ideas

NB: This blog post is adapted from one I wrote for an internal blog post at work, so it has been sanitised for content.

When learning new things, ideas, skills or exploring new perspectives, the image above reflects how I am feeling when I am trying to assimilate and process all the activities I take part in within the testing community. It’s that beautiful and terrifying moment when you are flying and the sun is ablaze on the horizon. It’s knowing you’ve learned a lot, but there is so much left to learn. Perhaps a move one way or another will lead to failure, but as long as you are quick to learn from those mistakes you can be on a well-lit path again.

Over the last month or so I’ve been attending a few events, including ITAKE Unconference in Bucharest, Romania, Let’s Test in Stockholm, Sweden and Nordic Testing Days in Tallinn, Estonia. Each of these events offered something new and different for myself as a contributor to those events. However, it is the other aspects of what they provide that are important.

ITAKE Unconference

ITAKE Unconference was my first time giving a Keynote talk at a conference. This was a huge honour not just to be asked to give a talk, but also the fact that it wasn’t a testing conference. ITAKE is a developer conference. Every attendee is highly technical, lives and breathes the code they write and the tools they use. It made me realise that whilst I have spent a lot of time learning about security, there is so much else to learn. Especially about how developers learn and work, and how applications are crafted.

I spent hours talking about how to build good environments for testing using tools like Docker and Heroku, or exploring how developers think about testing. A lot of it is about unit testing, some of it is about automation. But there are a lot of developers who understand the value of good testing and want to work with testers to make it happen. There is a lot we can do better to support them in this endeavour. These are things we should be doing at Medidata…testing cannot happen or exist in a vacuum.

Yes, there were those that think the role of testing or testers is now defunct, where a technical person can achieve all things they need to on a project. It was interesting to be able to discuss and challenge some of that thinking, where a tester can be a specialist or advocate for testing on their project; rather than someone who executes tests, gathers test results and creates endless meaningless reports. I’m not saying reporting is bad, just the doing the wrong reporting is bad, and unhelpful. It doesn’t add value, nor does it explain to those who don’t test what the value of the testing is.

ITAKE is a hotbed of software craftspeople. People who want to build and develop great software for their customers and clients. The best talk I went to while I was there was one of the other keynotes. Felienne Hermans, of the Netherlands, gave a talk called:

What is science? On craftsmanship for children

This reflected on her approaches to teaching coding to children. Children learn predominately through play, exploring their environment, and asking questions. It’s something that adults have largely forgotten how to do, or if we haven’t has become more formalised. We’ve turned play and learning into work instead. We can make our learning far more creative through events such as hackathons. We should review, model and landscape our applications inside the environment we are working in. Children do this far more naturally than (some) adults.

felliene1

 

She also talks about introducing children to the scientific method, how we observe behaviour, theorise about why the behaviour occurs and make a hypothesis, and then on to experiment in order to prove/disprove that hypothesis. This can be applied to coding as much as any branch of science or other learning.

smelly code

Let’s Test Sweden

This was a bittersweet event for me. This was my third visit to Let’s Test in as many years, but sadly it will be the last ever Let’s Test in Sweden. The organisers have decided to call it a day. This edition of Let’s Test had a distinctly technical focus, with each session being a three-hour workshop, held over two days.

I ran a workshop called Web Application Security, a Hands On Testing Challenge. We have Security Awareness Training in-house, which covers many of the techniques and tools of security testing, so this follows a similar path. Given the time and space in the office, we should be able to make this learning much more hands on. I try and provide a safe space for the attendees to find problems and ask questions about the application under test so that they can also talk with confidence about security in their own environments.

I attended a number of other workshops, including Alan Richardson’s Evil Tester’s Testing Games of Evil Testing. In this workshop, Alan introduced us to how we can use simple tools such as the browser developer tools to interact with simple JavaScript, API clients and HTTP requests to give us a competitive advantage in debugging web based games.

Whilst the topic might sound trivial, the application and usage of these tools are crucial for testing and debugging modern web applications. Browser based development tools will allow us to do so many useful things:

  • Viewing the source, both HTML and JavaScript
  • Debugging JavaScript, via the Console
  • Tampering HTTP requests
  • Testing REST services from the browser
  • Custom headers
  • Throttling application performance
  • Emulating other browsers, devices and screen resolutions

During the workshop, we played the games to understand them and their gameplay. We found bugs and fixed them, we wrote code and created cheats to make it easier to win, get massively high scores or infinite lives.

One of the other workshops I attended was Aare Nurm’s Pedal to the Metal. Aare hails from Estonia. His workshop was a fantastic, practical exploration of how software and hardware combine to make products, solving problems and exploring how stakeholder demands can cause issues and constraints with your testing, and how your testing value is perceived and acted upon.

We were given the exercise to test a keyfob for a new kind of car that was coming on the market. We didn’t have access to the vehicle, but only to a prototype to the keyfob. We needed to utilise all our testing savvy to come up with test ideas, find problems, analyse logs and even fix the issues ourselves.

arduino

Essentially the keyfob was a Raspberry Pi Zero, with a pin board and wires which could be configured to give different settings. We also had a set of LED’s which would flash according to the function the keyfob was supposed to be executing, including:

  • Unlock the car
  • Lock the car
  • Remote closing the windows
  • Activating the headlights
  • Remote boot/trunk opening
  • Start the engine

We initially observed the behaviour of the keyfob in order to determine it’s function. We were given minimal, but rapidly changing requirements not only for the product but also the business. Essentially if the car failed in the market, the company would go out of business. Hardware and software are so entwined, that even if the product is solid, well made and easy to use if the underlying software architecture is poorly implemented, this can result in a poor customer experience.

Here with my workshop partner Phil, you can observe our testing and learning. This was honestly one of the best learning experiences of my life. Check out the videos here:

Nordic Testing Days

I’ve run testing events before, such as South West Test in Bristol, and SWEWT (South West Exploratory Workshop in Testing). Never before have I helped to run a conference. Nordic Testing Days 2017 was my first adventure in being a conference organiser. It’s hard work, let me tell you.

nordic1

I’ve been lucky enough to speak at every Nordic Testing Days since 2014, on both using emotional heuristics in our testing, and security testing. Last year I decided not to submit again, to allow new voices to be heard. However, the organisers asked me if I wanted to be a part of the team for 2017. This responsibility came with all sorts of challenges, including organising the venue, social events, curating and selecting the content from the call for papers, interviewing the prospective speakers, organising and facilitating tutorials and talks, as well as solving logistical problems and finding replacement speakers for those that couldn’t make it.

chris1 grete1 fiona1
Christopher Chant

Friend and Volunteer

Grete Napits Marketing Manager/Chairperson Fiona Charles Keynote Speaker and Tutorial presenter

I had the pleasure of facilitating Fiona Charle’s tutorial The Art & Science of Test Heuristics. In this session, we were tasked with coming up with test ideas for two different scenarios, interspersed with both group activities and discussion in the round.

The first activity was to generate test ideas for a number of different puzzle games, such as SmartGames IQ Steps and IQ Fit. The task was to not only solve the puzzles but also identify and utilise heuristics for solving the problems that the puzzles posed. Many questions needed to be asked, including how many ways could we solve the puzzle, what problems or issues did we identify when solving the problem, what oracles can we use when solving the problems? It was no easy task, and one of the teams gave a massive cheer when they eventually solved their puzzle.

fiona2

The second activity was to generate test ideas from this video:

How many test ideas can you come up with to test the Oh Canada Beer Fridge? The main takeaway from this workshop for me was looking at heuristics as a tool to generate great testing ideas. It’s a complex problem, with no one size fits all solution. Test ideas are our life blood, by being the fuel for our learning and our ability to do our work.

 

Testing the testers: Let’s Test 2015 Reflections – Day 1

The night before

It is now almost a week since I arrived at Let’s Test near Stockholm in Sweden. I had heard a lot about Let’s Test, not least from my Weekend Testing colleagues Amy Phillips and Neil Studd. It was there this time last year that they decided to restart the Europe chapter. I had also heard a lot of good things about the conference from others in the community, all of which were overwhelmingly positive. So, as I recall my feelings and trepidations about attending and working at Let’s Test, I do it now with a renewed vigour regarding my career and learning.

The venue, nestled in a Swedish rural idyll on the Baltic coast close to Stockholm, is the perfect place. To say that it is beautiful is an understatement. The conference centre has the perfect combination of location and facilities that create a fantastic environment for learning, and of course, the socialising! After all, the conference is organised for testers, by testers.

Testers at the bar

Testers at the bar © Martin Nilsson / Lets Test Conference 2015
https://flic.kr/p/syAx7f

In addition to this challenge, I was not only running a workshop on security testing with Bill Matthews (more on that later) but I had also volunteered to be a facilitator. This meant that the workshops or talks I had volunteered for, I had to assist the speaker as much as possible with setting up and equipment, generally being a gopher for them. During the “Open Season” portion of the sessions, facilitators had to manage all the questions fielded by the attendees. The conference organisers had given us all K-Cards, to allow us all to take part fairly in the discussions. If you want to know more about K-Cards, check out this blog by Paul Holland – The history of K-Cards

Ben Simo -

Ben Simo – “there was not a breach, there was a blog’

Day One

The opening keynote was in a word, fantastic!

Ben Simo is a tester that I have been following for some time. His experiences and learning from attempting to organise health insurance on for his family would have been hilarious, if it hadn’t been so serious. “there was not a breach, there was a blog” was a fascinating journey through the issues and problems surrounding the release of healthcare.gov, the US Government website and initiative more popularly known as Obamacare.

Not only were there many functional, usability and performance issues with this site upon release, but also a huge range of potential security vulnerabilities. At the time, Ben blogged about these issues, trying to make the government aware of the problems and ultimately found himself somewhat reluctantly being the subject of media interest.

Ben is an eloquent and humorous speaker, who is extremely skilled and knowledgeable about his craft. His experiences also reflect strongly upon my recently learning in the sphere of security testing and as a result, the most significant takeaway I had from this talk was the matter of ethics when reporting issues in live, public systems. Ben emphasises the need to constantly be aware of the ethics of testing, and not harming the site. All in all, a brilliant start to proceedings.

Next up was an exciting and challenging workshop run by Emma Armstrong – “Equipping you for the unexpected challenges of testing”. I’ve known Emma for a while, but I’ve never seen her speak or run a workshop.

Emma Armstrong -

Emma Armstrong -“Equipping You For the Unexpected Challenges of Testing”

Emma had created a huge range of resources and a challenging application for us to investigate. Emma’s workshop encouraged us to examine and use a wide range and techniques and thinking in order to solve a testing problem. I really love pairing and working in groups with others, so this workshop really resonated with me. There is no better way to learn than to learn from others, in practical situations.

Emma’s enthusiasm, deep knowledge and skill in her craft is evident and clear from the content and presentation of the material. By examining and utilising thinking like Shneiderman’s Eight Golden Rules of Interface Design and Elizabeth Hendrickson’s Test Heuristic cheat sheet, we can overcome complex testing problems, without overwhelming ourselves. Using them as an oracle for any testing, where appropriate, then we can surely begin to equip ourselves for any unexpected scenario.

One of the best takeaways I had from this whole conference was during this session. I was pairing with two other testers, one from Sweden, the other from Romania. We discovered that our cultural differences, and in turn our similarities, often drive our thinking while testing. It’s not often I get to pair with testers from outside the UK, so this was a fantastic experience.

Our backgrounds and values often will impact the way we think about testing, and the problems we uncover – for example – a “Title” field would be almost unthinkable outside the UK, yet in the UK to not to be able to select whether you were Mr, Ms, Mrs, Miss or even a Captain or Lord would be equally strange.

After lunch I attended a half day workshop run by John Stevenson – “A Journey towards self learning”. I was facilitating this session, so helping out John with logistics and cold beverages! Despite my responsibilities preventing me from taking many notes, this workshop was and extremely engaging exploration of our own learning.

John Stevenson - A Journey towards self learning

John Stevenson – A Journey towards self learning

One of the major themes of the workshop was how constraints on information gathering can impact the quality of our learning and analysis of the information we gather. It can inform our opinions and how we apply values or biases to the learning we do.

One great example of this was a particular exercise. The group had to divide into three where each team had a particular task – discover as much as they could about the conference venue, with particular focus on the local flora. However each team had a major constraint imposed upon them – one was only able to use internet resources, another group could only use observations of the local environment, and the third could only speak to people at the conference venue. I went around with the third team to make sure the rules were adhered to.

The results were impressive and eye opening – whilst the team who had access to the web were able to gather a lot of data very quickly, they didn’t have the richness of data gathered by the other teams. It wasn’t easy for the other teams either, where it was fairly hard for team three to use information other than that gathered through word of mouth, as there was so much visual data to gather. Also, we were able to observe discrepancies and contradictions in the information that had been gathered. Its up to us as testers to be able to be mindful of our values and biases when analysing data, manage and work within constraints. John’s workshop was a fantastic way to engage with our own learning in an active and positive way!

All in all a fantastic start to an intense few days of learning! I’ll be blogging about day one and two over the next few days. Watch this space!

Critical Mass: A TestBash 2015 Preview

Hey testers!

Spring has sprung on the UK testing scene once more, as it is now seven days from TestBash 2015, held each year so far in Brighton. To those of you living under a rock, TestBash is the one day conference track and two day workshop run by the good people of Ministry of Testing, and especially Rosie Sherry. You can find out more here.

This year there are some established members of the testing community speaking, such as Michael Bolton, Iain McCowatt, Stephen Janaway and Matthew Heusser. I am looking forward to seeing these guys speak again, as they are always excellent, with insights and content beyond the conventional.

If there was a criticism of TestBash 2014 was that there wasn’t a diverse range of speakers. There were no female speakers last year, where now there are three; Karen Johnson, Maaret Pyhäjärvi and Sally Goble. Whilst I have read blogs and tweets by these testers, I’ve never seen them speak before so this is going to be incredibly exciting.

There are also new speakers to TestBash, such as Richard Bradshaw and Vernon Richards.

I’ve known Richard for a few years now, and he is an inspiring and knowledgeable tester. I’ve never seen him speak before other than during a 99 second talk. He’s the first guy I would go to for information on automation. He describes his talk as ““Test Automation” = Things don’t have to be this way”.

On to Vernon Richards, whose epic 99 Second talk on Myths and Legends of Software testing has been expanded into a full blown talk. Again, I have known Vernon for a while in the community. Being isolated down in the South West of England means that I don’t always get to meet testers based and working in the London area, but Vernon has been on my radar for ages.  Vernon’s 99 second talk last year earned him a huge cheer, and rightly so. This talk might turn out to be the jewel in the TestBash crown.

On to the workshops. Sadly I can’t attend the workshop day this year. With the TestBash workshops, it is your learning that is at the heart of it. The likes of John Stevenson, Simon Knight, Karen Johnson, Nicola Sedgwick and my Weekend Testing colleague Neil Studd all providing courses, it should add up to a fantastic day. Also running a workshop on BDD is Rikke Simonsen, who I had the pleasure of having lunch with last TestBash. Such a shame that I will be missing this fantastic opportunity to learn from them all. I’m definitely going to see if I can get in on that in 2016, as a learner or a trainer.

I cannot impress upon you enough the importance of TestBash in my career. I first attended in 2013. This was my first testing conference in three years, after what felt like some what of a period in the doldrums. I felt that I was coasting in my career and not doing enough to learn more, stretch myself creatively or professionally. I was just working.

A number of personal and professional events led me to attending that year, which gave me the kick up the backside that I really needed. As a result, I had my first speaking gigs in 2014. I am now speaking again at Nordic Testing Days this year, and visiting Let’s Test for the first time, running a workshop with Bill Matthews.

Sure, there are bigger conferences, with more tracks and a wider variety of talks, workshops and test labs, Some conferences are more popular with different testers, because of the variety of speakers and the depth and breadth of the content. However, what TestBash squeezes into only a few days in the compact and vibrant city of Brighton is phenomenal.

I’m also very proud to say that Brighton is sort of my home town. I grew up in a village not far away from there. This adds for me an additional pride and gratitude for the awesome effort that MOT and Rosie put into organising and running the event. As a result of the conference, and MOT as a whole, careers have been forged due to the community outreach and sponsorship of new testers so that they can attend courses and the conference for free, as well as other support. Some testers have even sponsored tickets themselves, which is hugely rewarding to the community. They should be thanked!

Two testers that are very important to me have so far benefited from this amazing community scholarship. Emma Keaveny has since moved to the UK from Ireland, secured her first testing role and along with Kim Knup have started to establish the first regular Brighton and Hove testing meetups.

The other was Danny Dainton, an ex infantry soldier, who actively pursued a career in testing after leaving the Army, and who I have the great honour of working with at New Voice Media. I really look forward to what these two fantastic testers do in the future, be it speaking themselves, or organising community events or just being able to work closely with them.

So, if you are going to TestBash next week, I look forward to seeing you there. It should be a fantastic event, full opportunities to learn and grow as a tester. If you want to talk to me, just grab me at Lean Bacon (ahem, Lean Coffee), at the queue for lunch, or at the Thursday or Friday meetups. It’s going to be EPIC!