Borland on Borland: Enterprise Agile Transformation

Borland just released a white paper that discusses our move to agile: Borland on Borland: A Case Study in Enterprise Transformation. The section on agile testing briefly covers the first phase that I’ve helped to define and implement:

One area where Borland was particularly challenged was around test automation. According to Agile principles, every feature that gets developed within a sprint must have associated test cases that have been run. Unfortunately, automating the tests is not always possible – and sometimes creates waste. For instance, if the user interface of a release is going to change significantly in a given sprint, any test cases that are created and automated will have to be scrapped and redone.
To overcome this issue, Borland made a slight adaptation to this Agile practice. In “Borland Agile” a feature or story is completed in a given sprint if the team has designed the test cases and run them to ensure they work. The automation is then completed in the next sprint. There are many risks involved in taking this approach. One of the tenets of Agile is that there is a clearly defined set of deliverables that must be met before a user story (or feature) is considered complete. By changing the completion criteria, and signing off on a feature or user story pending an action that will take place in the following sprint, there is the risk that the team will forget to complete the action – automate the tests – when they get focused on the next sprint.

The white paper only discusses one of several concerns with this ‘one sprint behind’ approach. The biggest concern, of course, is that the outstanding automation or enterprise testing tasks may uncover quality problems in the otherwise completed functionality. Therefore, at the end of the sprint, the teams that use this approach are careful to qualify what ‘done’ means. Typically, at the end of the sprint, the code is ready for user acceptance testing in specified environments, but we do not call it enterprise (release) ready until all outstanding quality tasks are completed.