Most Agile methodologies omit a crucial aspect of successful software development - Defining measurable business objectives for the initiative. When agile projects are aligned with well defined objectives they become a powerful business tool, without such alignment, teams can drift aimlessly from one epic to another, and often fail to deliver their intended business outcomes.
The Agile movement has revolutionized the development of software. It is clearly better than traditional alternatives. Organizations representing the full range of industries, business models, and sizes are using Agile. Interestingly, they aren’t all doing it the same way. There are many variants of the approach each with differing benefits. Overall, however, the approach remains robustly effective, despite significant variation in implementation.
Many of the practices and principles embodied in the Agile approach existed long before the Agile Manifesto (2001) codified them. In his 1988 book The Principles of Software Engineering Management Tom Gilb dedicated a chapter to Evolutionary Delivery in which he describes the critical concepts of of an Evolutionary Delivery approach as
These concepts overlap significantly with Agile Principles. The writers of the Agile manifesto cite Gilb as an influence, and Gilb himself quotes earlier work by Deming. Other examples of Agile-like approaches, stretching back many years, can be found in non-software engineering history books. But Gilb’s call for an approach that is objective-driven and results-orientated seems to have been ignored in favor of a focus on features over outcomes.
The collection of practices embodied in Agile and similar approaches has been co-evolving for many years. They work well together. These practices are starting to spread outside engineering into other areas of business. But this does not mean they are complete or perfect. They are merely a step on a path towards better engineering practices. In choosing how we might improve Agile it is important to remember what makes it so effective.
Each individual on an agile team can see how their contribution, and that of their peers, adds to the delivered solution. Agile teams are small enough that each individual can have a positive impact, and no individual can hide their weaknesses. Time scales are short enough for feedback to be meaningful and actionable. The team can individually, and collectively, learn from it’s mistakes, hopefully, fast enough to make a difference. People who like teamwork, and who like to learn, like Agile.
It is easy to communicate the basic principles and practices of the Agile approach. It is much harder to explain why these practices work. The benefits of the approach are emergent - practices interact in various ways to create benefits that are not predictable before hand. This is why so many zealots demand complete adherence to their flavor of Agile - they can’t explain how the benefits arise from the practices, because they emerge from a complex interaction of many factors. The short feedback loops inherent in the approach create many opportunities for correction and improvement. In reality it doesn’t much matter where you start. Not only do the practices lead to emergent benefits, they act as an attractor that pulls and pushes teams toward an ever more effective approach. The practices are valuable individually and are self-reinforcing, becoming even more valuable collectively.
Agile can be employed to evolve better solutions in a rapidly changing environment. But there is more than one reason why a team may lack a clear understanding of what it is trying to achieve.
When a novice team is uncertain of how to deliver a well defined solution an agile approach forces the team to continually confront their own lack of experience, to fail in small recoverable steps, thus allowing them to succeed in the longer term, if they can learn fast enough. The approach allows investors and other stakeholders to keep inexperienced teams on a short leash while they learn how to deliver effectively. Agile is a great way to reduce delivery risk caused by inexperience. Agile forges self-calibrating teams that know the limits of their own capabilities.
Conversely, agile forces expert teams to confront their lack of understanding of the complex problems they are trying to solve. Agile helps expert teams explore the problem space by allowing them to innovate in small chunks and incrementally evolve a solution. Evolution is a fundamentally wasteful process as Tom Gilb has said “rapidly iterating in wrong direction is not progress”. When guided experts, an evolutionary approach can be the best way to explore a new problem space.
In practice most teams are experts at some things and beginners at others. This ability to help both Novice and Expert teams incrementally improve and create better solutions is the key to the success of Agile.
The conditions that favor the Agile approach do not universally apply. The HBR Article Embracing Agile points out it is important to “understand where agile does or does not work.” Not all initiatives require innovation. Some merely require a solid implementation of a straightforward design.
Agile is not a panacea. It is most effective and easiest to implement under conditions commonly found in software innovation: The problem to be solved is complex; solutions are initially unknown, and product requirements will most likely change; the work can be modularized; close collaboration with end users (and rapid feedback from them) is feasible; and creative teams will typically outperform command-and-control groups.
Even when Agile is applicable, there are multiple flavors, each suited for different purposes. Disciplined Agile (DA) is a process decision framework that provides guidance to organizations struggling with deciding which Agile Practices to adopt.
In general, Disciplined Agile (DA) is a hybrid framework that builds upon the solid foundation of other methods and software process frameworks. DAD adopts practices and strategies from existing sources and provides advice for when and how to apply them together. Introduction to Disciplined Agile
With an objective expert assessment of the needs, capabilities, and constraints of a business, and the problem it is trying to solve, it is usually possible to tailor a successful Agile approach. Tailoring the approach is not that easy, it requires experience, but Disciplined Agile (DA) does a reasonable job of providing guidance.
Despite the variety of agile approaches available, few define a practices concerning business outcomes, yet outcomes matter far more than features. Agile is silent on how to define and use objectives to guide project execution. There is no principle or practice regarding this obvious need. Agile is mute on how objectives can be used to filter and interpret user feedback. Not all feedback should be acted upon, if it doesn’t align with project objectives, then maybe it should be ignored? Most concerning, Agile has nothing to say on how to address the omni-present business objective to deliver a viable set of features, by a particular date, for a pre-approved budget. In fact many Agile adherents would claim this is the wrong way to frame the problem. The best that most Agile methodologies can do is to offer a trite promise to deliver “value” without defining what that is or when it will be delivered.
What is needed is a light weight objective definition and tracking practice that can be incorporated in Agile projects. Fortunately this exists. Filipe Castro has done a nice job of explaining the OKR (Objectives and Key Results) approach to goal setting. He further explains how OKRs can be used for Agile Development.
OKR (Objectives and Key Results) is a goal setting framework adopted by Silicon Valley companies that can complement Agile and Lean, creating an agile approach for setting goals and measuring value. Used by Google, Twitter and LinkedIn, OKR can help evolve the Agile Community and the Agile Manifesto itself from a focus on outputs (delivering features) to a focus on outcomes (delivering results). Agile Goal Setting with OKR - Objectives and Key Results
Using OKRs to define the business outcomes for a project is a practice that most Agile teams would benefit from. Objectives are NOT the deliverables, they are the beneficial outcomes the deliverables will produce. There need only be a few - 3 to 5 at most, with 1 to 3 key results per objective. Objectives should be shared with the team and stakeholders, they should be printed out and posted on the wall. They become the team’s objectives providing guidance in times of confusion and doubt.
Some examples may help solidify the concept.
For a medical imaging solution the following three product claims guided engineering efforts:
These claims were public and contractual. They helped differentiate the product in the market and were treated as objectives by the engineering team. They influenced many engineering decisions on a daily basis.
A recent project to deliver an application development framework for consumer devices defined some objectives as follows:
It would have been easy to develop a framework that had all the required functionality but failed to deliver the business requirement for speed of development and ease of adoption. These objectives helped the team focus on the real business value.
For a sports related social network application.
This last objective kept a small team fully occupied leading up to the launch. They designed and built a distributed test harness to emulate high volume traffic, implemented and tuned a Redis cache with event based de-caching, modified a few other services to remove bottlenecks and were able to demonstrate the performance characteristics of the system up to the point at which it failed. They exceeded the requirements by a modest amount and were able to identify the maximum load after which performance would start to degrade. This enabled the business to make strategic decisions about hosting and burst capacity planning.
Once a team has taken ownership of the project it is up to them to manage changes to the objectives over the lifespan of the project. Inevitably objectives start to drift as the project proceeds. Sometimes valuable lessons are learned along the way and objectives can be reset to take advantage of our new understanding. Sometimes externalities move the goal posts and the project must adjust to the changing business circumstances. Sometimes objectives that were originally excluded are added back into scope. Whatever the case, having defined objectives allows the team to monitor such drift, managed it, and stakeholder expectations, in a controlled way.
Without defined objectives teams can drift aimlessly at the whim of changing circumstance and fickle stakeholders, with well defined objectives teams can ensure they stay on track to deliver the required business outcomes.