Damon Poole points out a benefit of agile development that I hadn’t given much thought to:
…the reason I enjoy Agile development is because it made my job more fun. Today, thanks to Agile development, I interact with customers more than ever before. As a product owner, I do more demos and am able to provide new features that hit closer to the mark faster and more frequently than ever before. This in turn means more oohs and ahs from customers which is more fun for me and more profitable for the business.
As I mentioned in an earlier post, I’ll be working the Borland booth at the Agile 2008 conference in Toronto next week. We will be showing the integration between SilkCentral Test Manager and a popular open source testing tool. (The marketing people won’t let me say be more specific here; come by the booth at the conference to find out more!) The guy who’s preparing the booth demos and materials for the integration just sent me the following email:
Iâ€™ve been working . . . on our Agile Test Strategy for SCTM [SilkCentral Test Manager] and how it relates to our other solutions. Iâ€™m putting together the PowerPoint for the ‘demo loop’ in the booth based on the information I was able to gather over the last couple of days.
Also, I have a VMWare image of the integration . . . I plan to have screen shots of the demo . . . but have the VMWare image available for those whoâ€™d like more info on the integration.
The integration that we will show at the conference was developed very recently but will be released later. Due to our internal use of agile, we have this functionality working now, and we will be able to demo it. Without agile, I would have been stuck in the booth with the slide deck and other published materials only.
I just ran across an old article by Scott Ambler in Dr. Dobb’s Journal that provides a good introduction to agile testing.
I’ll be attending the Agile 2008 Conference in Toronto next week. Among other things, I’ll be working the Borland booth a few hours each day. Drop by and see me. And while you’re there, watch our ‘Day in the life of an agile team’ video, in which I star as … wait for it … an agile team member. I have a speaking role and at least a couple of minutes of screen time!
My coworker who has a two-letter last name has complained of applications that require a longer string in the ‘last name’ field. Somehow, I think most applications I’ve ever tested would have the opposite validation error for this guy.
Lately, I’ve been thinking a lot about the role of conventional QA engineers on agile projects. In my earlier post on this topic, I concluded that there is less need for non-coding QA specialists on agile teams, but that the skills that such a person brings to the table are still necessary.
In a recent column over at StickyMinds.com, titled Does Exploratory Testing Have A Place On Agile Teams?, Johanna Rothman writes the following:
Agile projects require true generalists as testers: people who have requirements-, design-, and code-understanding skills. Without those skills, they can’t think critically enough about the product under development, and they might not be able to create enough variety of tests. If they understand the requirements, design, and code, they can turn that understanding into crafty tests. Some of those tests will be exploratory. Even some of the exploratory tests will need to be automated in order to repeat them. And, I’ve seen great testers on agile projects who can quickly create automated tests to do some of their exploration.
I read that as a confirmation of my thinking. Coders can indeed do testing, but it takes someone on the team who has the big picture in mind in order to do good exploratory testing. QA generalists are needed to do this work.
In a recent blog post, Tobias Mayer observed that one of the hottest topics around is how to do agile with geographically distributed teams:
Many Scrum practitioners these days are working hard to come up with the best way to make Scrum work in distributed (usually off-shore) environments. There are many articles being written on this topic, and many submissions to the major Scrum and Agile conferences. They all say something similar, we know that co-location is ideal, but the reality isâ€¦ and then go on to offer â€œsolutionsâ€ (read work-arounds) for the non-ideal, real world situation where team members are scattered around the globe.
Tobias’ main point is that we should question the assumption that we have to work with distributed teams:
It is interesting how the phrase â€œthe real worldâ€ is used almost as a weapon to wield against the idealist: yes, it is all very well to say that teams should be co- located but that is not how people actually work in the real world. We seem to forget that reality is not something that is imposed on us. Reality isnâ€™t just there; we create it. The phrase â€œthe real worldâ€ is akin to the phrase â€œitâ€™s just the way we do things around hereâ€. Both should be challenged with a healthy dose of skepticism and a steady barrage of â€œwhyâ€ questions. Just because it is, doesnâ€™t mean it has to be.
From my experience, I think Tobias is absolutely correct on both counts. Whenever I’ve heard someone ask about making agile work with distributed teams, the first response is that it can’t; teams need to be collocated. And invariably, the person posing the question replies that the decision to have a distributed team is out of their hands. And in my experience, most of the time, the decision to embrace off-shore resources really is out of the practitioner’s hands. The practitioner can question the wisdom of distributed team all he wants to no avail.
But you know, this is just a subset of the general issue with offshore development. The exec who makes the decision sees money: I can hire two engineers in X for the cost of one engineer in Y. But the practitioner has to deal with the reality: two engineers geographically removed from the rest of the team are hardly as productive (or less so) than one collocated engineer. Plus, the company is paying gobs of money to fly people around the world to manage the arrangement.
As the outsourcing situation matures, my prediction is that decision makers will begin to understand the actual value proposition of outsourcing, and we’ll see more collated teams again.
If the problem lies with distributed teams, not with some fundamental disadvantage of off-shore development itself, then the people advocating collocation had better be prepared for this eventuality: their company may just decide to collocate teams by moving an entire development effort off-shore. I know of one company in Austin that has moved all of its development to India, and my own company has relocated all development for one product from the US to Singapore.
In an earlier post, I compared agile to 12-step programs, and I listed 12-step slogans that could also apply to agile.
Since I posted that, one slogan that I missed keeps coming to mind: Fake it till you make it.
In AA, ‘Fake is till you make it‘ is “a suggestion often made to newcomers who feel they can’t get the program and will go back to old behavior. The suggestion implies that if the newcomer acts according to the steps and teachings of the program, then the program will begin to work and the anxiety will fall away.”
In his book Agile Software Development, agile guru Alistair Cockburn applies the Japanese martial arts concept of ‘Shu Ha Ri‘ to agile adoption. Mr. Cockburn writes of the first phase: “In Shu, the student should be working to copy the techniques as taught without yet attempting to make any effort understand the rationale of the techniques of the school/teacher” (p. 18). In the second phase, Ha, “the student must reflect on the meaning and purpose of everything that he has learned and thus come to a deeper understanding of the art than pure repetitive practice can allow” (p. 18).
To me, that sounds an awful lot like ‘Fake it till you make it.’
Borland just released a white paper that discusses our move to agile: Borland on Borland: A Case Study in Enterprise Transformation. The section on agile testing briefly covers the first phase that I’ve helped to define and implement:
One area where Borland was particularly challenged was around test automation. According to Agile principles, every feature that gets developed within a sprint must have associated test cases that have been run. Unfortunately, automating the tests is not always possible â€“ and sometimes creates waste. For instance, if the user interface of a release is going to change significantly in a given sprint, any test cases that are created and automated will have to be scrapped and redone.
To overcome this issue, Borland made a slight adaptation to this Agile practice. In “Borland Agile” a feature or story is completed in a given sprint if the team has designed the test cases and run them to ensure they work. The automation is then completed in the next sprint. There are many risks involved in taking this approach. One of the tenets of Agile is that there is a clearly defined set of deliverables that must be met before a user story (or feature) is considered complete. By changing the completion criteria, and signing off on a feature or user story pending an action that will take place in the following sprint, there is the risk that the team will forget to complete the action â€“ automate the tests â€“ when they get focused on the next sprint.
The white paper only discusses one of several concerns with this ‘one sprint behind’ approach. The biggest concern, of course, is that the outstanding automation or enterprise testing tasks may uncover quality problems in the otherwise completed functionality. Therefore, at the end of the sprint, the teams that use this approach are careful to qualify what ‘done’ means. Typically, at the end of the sprint, the code is ready for user acceptance testing in specified environments, but we do not call it enterprise (release) ready until all outstanding quality tasks are completed.
I just ran across a blog post by Tobias Mayer that compares scrum to Alcoholics Anonymous. Like scrum teams, AA groups are self-organizing with no formal leadership, and the participants of the group work closely together on a focused, shared goal. And of course, both AA and scrum require constant re-evaluation and change. Tobias goes into more detail in his blog post.
To extend this line of thought, I reviewed AA slogans and found many that would apply to scrum. Among them:
- One day at a time
- First things first
- If it works, don’t fix it
- KISS–Keep it simple, stupid!
- Live in the now
- Be part of the solution, not the problem
- Decisions aren’t forever
- Let go of old ideas
- Change is a process, not an event
- Courage to change
- Principles before personalities
As I learn more about and gain more experience with testing in an agile environment, I’m becoming increasingly concerned about the suitability of conventional QA engineers in agile environments. Since agile methodologies stress that the entire team is responsible for testing and that as much testing as possible be automated, the QA specialist has limited usefulness on an agile team.
I still believe that agile teams need people whose primary interest is ensuring quality. These team members should be primarily responsible for some traditional QA tasks: recommending the types of testing that need to be performed, recommending and/or planning testing strategies, tracking that all the necessary testing is performed, etc. But on an agile team, not all of the test writing and execution itself is necessarily performed by these same team members.
So, what does this mean for conventional QA engineers?
When asked for career advice, I’ve always recommended that QA engineers become as technical as possible: learn programming languages, testing tools, etc. Quality assurance in an agile environment just strengthens the necessity of this recommendation. I don’t think it’s necessary for QA specialists to become capable of writing production code (though it’s helpful!), but a more technical QA engineer can work on a larger portion of the automated test architecture, test writing and execution.
I’m interested to hear others’ thoughts on this.