Acceptance is at the heart of every consultancy. We don’t deal in tangible deliverables like a crate of carrots that are either delivered or not. We make a promise to a client every time we engage in work that our outputs will deliver a business benefit. For the client, the payback could be immediate or it could be several years down the line.
In most cases we ask the client to pay us at the time the work is done, so there must be clear deliverables that the client can verify and sign-off before they part with their money. How do we square that reasonable expectation with the difficulty in predicting the course of Agile software development?
Thankfully, Agile provides a mechanism for this: Acceptance Criteria. I’d like to talk a bit about the journey we’ve travelled making acceptance criteria work equally well for us and the client.
The whole is the sum of the parts
A simple truth we learned about acceptance on a large build is that defining acceptance criteria for the system as a whole is unworkable. Just as we break down requirements into small units of work (user stories) we define acceptance criteria at the story level too. It makes sense to build a system as a combination of user stories, and by the same logic we can say that if each story is acceptable then the system can be acceptable as a whole, right?
Wrong. That’s like saying that if we get 4 wheels, a chassis, an engine, a body shell and bolt them together then we will have an acceptable car.
Both the client and the engineers need some assurance that the whole system will be integrated neatly and we tackle this with a special category of acceptance related to integration criteria. User stories define what will be built and integration criteria define how it will be put together. This is similar to the traditional distinction between functional and non-functional requirements.
At SSA our focus is almost solely on web and mobile applications so these type of criteria are pretty common from project to project. A typical set of integration criteria would include:
- The application must be hosted online and accessible from anywhere via a URL
- The application must only be accessible to authenticated users via username and password
- All screens of the application must be protected by an SSL certificate with 128 bit encryption
- All functions of the application must be accessible from conventional web menu navigation
So we say that if all functional user stories meet their own acceptance criteria and the system as a whole meets the integration criteria then the system is acceptable. This makes sense and by and large clients are happy to go with that.
This brings us on to the thorny issue of drafting acceptance criteria. How to write a description of a function that has not been built yet that the client is willing to sign-up to and the engineers are willing to give committed estimates for? Let’s not forget that this is a commercial transaction so all parties are being cautious about what they agree to!
In the last few years we’ve gone through a few different approaches to try and get this right:
Attempt 1: The Novel
The temptation is to write down as much as possible to cover all possible aspects of this hypothetical feature. The result: lots of time writing specifications that become superseded as soon as development starts, a reduction on the proportion of budget that is spent on building features, and a reliance on abstract descriptions of features rather than the concrete features themselves. This is the logical result of Waterfall development and is most un-Agile.
That’s not to say that detail is not sometimes required and I’ll look at this in more depth in a future blog on ‘elaborating user stories’. Suffice to say that the documentation should be as short as possible… but no shorter!
Attempt 2: The Standard Format
After a bit of research we decided that we would try a well-known standard format for Agile acceptance criteria, much like we do for user stories. The format is as follows:
Given that [the system is in a certain state] when I [do a certain action] then [a result happens].
We found this very appealing in theory as it has a nice symmetry with the format of a user story but in practice we found this quite unworkable. The problem is that stories can be very different in nature and criteria suitable for a contact form submission is very different from criteria suitable for a commodity trading algorithm.
We soon dropped the standard format idea as elegant in theory but impractical and artificial.
Attempt 3: Who needs acceptance criteria anyway?
Our next approach (by default after the consensus failure of the standard format) was to stop using acceptance criteria altogether for a period of time. This actually worked ok most of the time but more by luck than judgement.
Then we signed up to build a back office system for a financial company and decided that it probably wasn’t very sensible to have our delivery risk hinge on whether our interpretation of the user story was the same of the client’s… after the development cost had been incurred!
We were saved from our quandary by a talented Agile BA who worked with us for a while (thank you, Divya) who pointed out that the easiest way to draft acceptance criteria is to ask the client what they want!
Attempt 4: Client-led criteria
This is simple genius: the BA will simply ask the client: “So for this user story, Mr Smith, what do you have in mind for the solution?”
If the client is web-savvy then you will probably get something very useful that can be the basis of the acceptance criteria, almost verbatim. It is not unusual for a client to say something like:
“Well, I’m thinking that we can add another menu item over on the right here, called ‘audit’ and it takes you to a page showing all the users that have logged in in the last 7 days with the date, time, and username. Then maybe when you click on the user’s name you go through to a list of all the pages that they visited in that session”.
If the client is not so web-savvy then the BA suggests the solution in similar terms but asks for confirmation that the client likes the idea. So the BA says something like: “Well, what we’ve seen work well in other systems is… (description of suggested solution)… How does that sound, Mr Smith?”
The client will often add a few tweaks (“I think the link would be better on the left than the right”) to make it their own, but in majority of cases you have your acceptance criteria right there. As important bonus is the client’s buy-in from the outset because they have either defined the solution, or agreed the solution, in a co-creation process.
So at SSA we recommend a client-led approach to acceptance criteria on the basis that the client knows best what they will be happy to pay for. This does come with the caveat that we are the consultants and we are the technical experts so the client will depend on us to guide them to a satisfying solution, but in bespoke development the clients tend to self-select as tech-savvy so their input is often very helpful.
As a web development team there is nothing more satisfying than showcasing new work to a client and seeing a delighted response. It touches a very human need for approval and acceptance… and keeps the Finance Director smiling too!
- Structured Agility: Define Scope, part 2 (theadaptivepm.wordpress.com)