What is Agile?
Agile was born in the world of software development, where you can build the first version and keep adding to it or changing it to make improvements. You get to see results more quickly and because you can check and change what you do in each iteration, it lowers the risk of getting it wrong. It is an approach that accepts uncertainty and promotes experimentation and close collaboration with customers.
The Agile approach is often contrasted with Waterfall methodologies, where development flows through a series of phases that can take months, if not years, to deliver.
You can read all the articles in our Product Management Journal – Agile by signing up for free here.
The challenges of Waterfall
The challenges of Waterfall come from the need to define specifications in detail up-front (when there is often uncertainty about exactly what’s needed) as well as the time it takes to get from concept to delivery.
Stakeholders, including customers, don’t get to see what’s being built until the very end of the process. There’s a significant risk that resources get wasted building requirements that were misunderstood, ‘gold-plating’ (over-engineering what’s built), or delivering something that simply doesn’t work for the stakeholders.
Also, during the long build phase, customers evolve their thinking, competitors launch products, and the market can change. These change what the new product needs to be. Some requirements may no longer be needed or new requirements may need to be squeezed in so the product won’t fail when launched. It’s easy to get things wrong.
The attraction of Agile
Agile approaches address these challenges by minimizing upfront planning and documentation, shortening the delivery cycle, and engaging customers (or customer proxies) frequently during the development process. These make it possible to quickly release product versions and validate that they deliver what’s needed. If successful, then the initial release can be extended with small iterations. But, if the initial product doesn’t fit the market need, then the decision can be made to stop investment or ‘pivot’ to a better solution. Even if the first product is a failure, the cost to the business is low because of the short release cycle.
The mantra is… if you’re going to fail, then fail fast, so you can learn quickly and either stop investing or change to something better.
Is Waterfall dead?
Waterfall or stage-gate processes still have their place. Waterfall may still be the best option when requirements are well defined (such as when building a standard API) or when there are projects with complex and interconnected hardware and software components.
Fig. 1 Agile vs Waterfall development approaches
Even in these cases, most companies that use Waterfall are pushing for shorter release cycles. It’s not uncommon for companies who previously released every 2-3 years to shorten their cycle to every 6 months – 1 year. This gives some of the benefits of an Agile approach as there is the possibility of releasing smaller, less costly increments to validate that the product meets market needs.
It’s also frequently the case that organizations that do Agile development have governance that uses a gated process. This enables senior stakeholders to validate whether planned outcomes are being achieved before committing further funding.
Agile adoption
There are very few product people who are unfamiliar with the concepts in Agile; however, its adoption is not universal. From our 2020 annual survey, we know that 35% of companies use pure Agile, 30% use a hybrid approach, 24% use Agile or Waterfall depending on their product, and 11% use Waterfall only.
We’re Agile but…
Fig. 2 Approaches to development
When product people are asked about which approach is used, they often say ‘I’m using Scrum, but…’ This is because most, correctly, customize and adapt Agile practices such as Scrum to fit with their organization’s existing processes and the type of projects they are working on (and in some cases, various organizational and behavioral malaises mean that they’re really Agile in name only).
The hybrid approaches have colorful names, for example, ScrumBan is a mix of Scrum and Kanban. Scrum-fall, WAgile, or Water-Scrum-fall are where Scrum is used during development, but the surrounding business processes for market analysis, business cases, launch, etc. are Waterfall.
A 2020 survey of Agile methods by VersionOne identified the most common Agile approaches – these are shown in the pie-chart below.
They affect, for example, how requirements are identified, prioritized, and communicated. The frequency with which what’s been built will be shared and approved, as well as the processes for releasing to a live environment ready for customers to use.
Making Agile work in a business
Agile software development techniques have been around since the 1970s and the Agile manifesto was first created in 2001. With this long history, many organizations have grown up using an Agile development approach.
However, if a company does move from Waterfall to Agile, it is usually initiated by the development team and they typically go through major re-tooling and process improvements to enable the shift. In either case, to make Agile work, the development team need close collaboration with customers (or customer proxies) to write and prioritize requirements and to validate that what’s built meets the market requirement. This is where Product Owners and Product Managers play their part – managing requirements, signing off on work done, and answering questions about customers.
But, there is a fundamental clash in many businesses. Agile by its very nature means adapting to change. It is therefore difficult to predict where you’ll get to and by when.
In contrast, to run a business, senior management wants to know when a product will be ready, how much it will cost, and how much it will sell. In addition, there are many other parts of an organization that are critical to creating a successful product. These teams, such as procurement, legal, and marketing may be doing activities that have a long lead time. To optimize their workflow, they need to know well in advance what’s needed and by when.
Conclusion
Every business wants to be able to do things more quickly and flexibly (to be agile) but implementing an Agile development methodology is only one part of the answer.
Organizations adopt Agile at different levels and in different ways. Some are born using Agile and some are still experimenting. Some have a small Agile team, others use it for all their development. And, a few have fully embraced an Agile approach across the whole company.
If you listen to some advocates, you could be forgiven for thinking that Agile is a ‘silver bullet’ which can solve all your company’s problems. But every business has deadlines, commitments, constraints, and an established company culture. And most businesses have existing products, customers, processes, infrastructure, and teams.
All this context impacts how Agile can work in an organization.
Agile methods in brief
Scrum – time boxed delivery is the most widespread and popular Agile software development approach. It defines key roles and a project delivery framework that delivers short, time-boxed, incremental, development releases called sprints.
Each development phase (sprint) runs for a set number of weeks. This limits the scope of what is agreed for delivery during the sprint.
Kanban – capacity based delivery is an alternate to Scrum that optimizes workflow. Whilst Scrum is time-boxed, Kanban limits the amount of work allowed in each stage of a build. For example, imagine a prioritized queue of requirements waiting to go to an ‘In development’ stage of build followed by an ‘In testing’ stage that validates what’s been built. A limited number of requirements are allowed in each stage. Once capacity becomes available then additional tasks are ‘pulled’ from the previous stage. There is no time limit on how long each requirement should take to complete – though, in the interests of agility, each will typically be sized so they can be completed within days or weeks.
Development/test don’t start working on something new if they are already working on an agreed maximum number of requirements.
Dual-track Agile – one team, two tracks refers to a way of working where product discovery ( see article here) and product delivery are happening at the same time. Discovery is a continual process that identifies opportunities that can then be validated as the “solution space” is explored with customers. Once ideas have been validated through Discovery they are fed into the Delivery track. So, Discovery fills the work backlog and Delivery empties it.
Explore ideas and run activities to uncover insights into what to deliver. Agile development to quickly get to a product/market fit.