Lately, I’ve been thinking a lot about the role of conventional QA engineers on agile projects. In my earlier post on this topic, I concluded that there is less need for non-coding QA specialists on agile teams, but that the skills that such a person brings to the table are still necessary.
In a recent column over at StickyMinds.com, titled Does Exploratory Testing Have A Place On Agile Teams?, Johanna Rothman writes the following:
Agile projects require true generalists as testers: people who have requirements-, design-, and code-understanding skills. Without those skills, they can’t think critically enough about the product under development, and they might not be able to create enough variety of tests. If they understand the requirements, design, and code, they can turn that understanding into crafty tests. Some of those tests will be exploratory. Even some of the exploratory tests will need to be automated in order to repeat them. And, I’ve seen great testers on agile projects who can quickly create automated tests to do some of their exploration.
I read that as a confirmation of my thinking. Coders can indeed do testing, but it takes someone on the team who has the big picture in mind in order to do good exploratory testing. QA generalists are needed to do this work.
I disagree that explorers MUST BE testing generalists. Exploratory testing involves two concepts: testing and exploration.
Testing is an empirical search for information about the quality of a product or service. We can do testing in a scripted way, that does all the thinking during test creation and mechanizes the execution, or we can do testing in an exploratory way (blending thoughtful design, research and execution into an always cognitively-active process). Obviously, real-life testing is often done somewhere between these extremes, as a blend of activities of different styles. Similarly, testing can be done with different scopes, from system testing through unit testing, and the style can still range from exploratory to scripted. And we have a wide array of tools to do computer-assisted testing–no one knows how to automate testing, we only know how to automate some parts of the task. The tools can support exploratory or scripted activities.
Programmers understand many of a project’s risks. They are probably better equipped to create thoughtful tests to explore those risks than non-programmers.
Other people are more focused on the integration of the software within its environment. They understand devices, communication protocols, cross-platform variables that trip unprepared code, etc. Some of these folks are programmers–some of them write interfaces between the application and the environment. Whether they are programmers or not, these folks are specialists too–I’ve worked with several–but they understand a different body of risks from typical programmers and from typical system-level (user-needs oriented) testers.
Similarly, we know specialists in performance evaluation and security evaluation. Some are programmers, some not.
All of these folks, along with the system-level software validators, can test in a scripted way or an exploratory way.
Any tester can explore. Any programmer can do exploratory testing. But people will differ, as they explore, in what risks they explore, what tools they will use, what techniques they will apply, and thus what they will learn.
I think it is as much a mistake to insist that programmers cannot or should not do high-skill exploratory testing as it has been, within so much of the agile community, to denigrate the idea of skilled testing by nonprogrammers and the idea that there could be “generalists” on a project who were non-programmers. These walls are artificial and the prejudices they foster are counterproductive.
We can say skilled testing generalists can add value to a project without insisting that no one else can do any of what they do or understand what they understand.
— Cem Kaner