July 13th, 2009 tkadom
Recently I was in a meeting with a group of QA managers of a fortune 500 company which had decided to take the agile leap. The question before the managers was: “How will our QA processes map to an agile environment? How do we interact with Agile development teams, and how can we ensure that we deliver a quality product? This was no small feat, considering these managers represented several shops over a dozen product lines, with a diverse international staff of consultants and full timers. The challenge was not just shaping the strategy for moving to agile, but also identifying the tactics that would make that strategy a reality.
Due to the time constraints we decided that the meeting should first focus on strategy, and the tactics would fall out of the strategy or even be left for another day. After we got over some of the basic conceptual hurdles of what it means to be agile, and the QA managers accepted the notion that there would be one team rather than two federated competing departments, we landed on the definition of “done”. The general consensus was that it was not done until it had completely passed all system and integration testing. The group lamented how development would just throw stuff over the wall and label it as done to collect the points for velocity.
In their initial attempt to combat this behavior, they decided on multiple states for “Done”. They would have a “development done”, a “user acceptance done”, and a “QA integration done”. The “user acceptance done” would be enough to satisfy the requirement that a story met the basic acceptance criteria and the points for completion could be awarded, but it would then be fed into the much harsher QA filter where it could ultimately still be rejected. From the QA perspective this was the only way to ensure that the quality was in the product, and since their behinds were on the line for the quality, they would not accept anything less. Due to the additional quality filter, their agile environment demanded that all stories for an iteration be delivered early so that full regression testing could be done prior to iteration end. A waterfall smell.
So the conversation shifted to who was responsible for the quality of the product in an agile environment. Was it the QA team, or the development team? Clearly the quality responsibility for a self directing self organizing team would have to be on the development team, and not on QA unless QA was embedded as part of that team. However, lacking embedded QA, If the team took responsibility for the quality of the product, would the QA managers then be able to lift the toll gate entirely for product quality?
The answer was a unanimous “yes… but”. The “but” obviously being tied to the failings of developers to test anything properly or consistently. “Developers do not know how to write proper tests”, and “Developers do not have access to full integration tests, and therefore their testing can only be limited to their code”. Were common refrains. Putting those arguments aside for a moment, in an agile environment, the product owner or client ultimately agrees to accept software from development. The argument basically boiled down to product owners and clients not having the “know how” or “white box understanding” to properly “accept” a story.
That was certainly a worthwhile point to arrive at, because we had established that if the development team and product owner truly owned the “Quality” of the product, then the “Assurance” portion of the QA title was no longer relevant. What was now “QA” would then be something minus the “assurance”, but what would that “Q” look like?
“Q” would have to work directly with the customer in accepting stories so that stories could be accepted based not only on meeting the acceptance criteria, but on how they integrated with the system, and how they met the more stringent guidelines for acceptable use.
The problem with this definition was basically the time to acceptance. Acceptance of a user story should not take a great deal of time. Certainly not more than a day, and if at all possible story verification should be near instant from the customer and developer perspective. Clearly system testing needed to be automated, and regression tests had to be readily available to be triggered on story delivery.
This finally landed the QA managers where I felt they needed to be. On the customer side accepting stories, automating acceptance, and regression testing. What I did not realize until today is that while this concept was basically agreed on by the managers, it also redefined the department and the meaning of “QA”. There are many agile presentation slides that kick the building down on QA. QA is dead in agile, QA does not exist, QA is a roadblock. This is clearly not true. QA still exists, but the function is split. The assurance belongs to the team, and the quality decision belongs to the product owner.
However, since the product owner is unable to fully determine the quality risk, QA slips into the role of “Q”, or to avoid any references to star trek, perhaps “Product Quality(PQ)”. “PQ” is tasked with working closely with the customer or product owner to ensure that the conversations in the user stories are accurate and that the product still works beyond those stories. “PQ” is a filter, not a gate. The product owner can still overrule PQ, and the PQ manager will not get fired if the product owner does so to the detriment of the product. Renaming QA is better than killing it. The function still exists, but the responsibility shifts.
Tim
Posted in Architecture, Random | 2 Comments »
