Agile vs. Waterfall: What's the difference?

People often compare waterfall vs agile and argue one is better than the other. Discussions also happen around simple requirement vs complex requirement and if simple then waterfall and if complex then use one of the agile approaches. Is it that simple? Is comparing based on requirement type correct?

Another argument is about predictive vs empirical and if predictive then waterfall else agile approach. People also do the comparison of project vs product. If you have project (often referred as fixed time and fixed cost) then go for waterfall and if you have a product that is being built based on market reaction then take one of agile approaches. I don’t see people consider full context and that stumps me. Let’s talk about complex predictive project such as setting up disaster recovery system, automation of plant, rolling out SAP HR in 28 multiple countries simultaneously or automating delivery pipeline.

Here we have predictive work like setting up production like environment for DR that should work if needed as expected, automation of plant and automated delivery pipeline to improve quality and productivity, adopting lean HR practices across company through SAP HR. What approach would you suggest in these scenarios? Should it be waterfall or agile? Can we have incremental release for all these if we choose agile approach? If yes, will these small releases be readily available within a month? If no, then should we take a waterfall approach? Or should be try WaterScrumFall, ScrumBan or WaterKanban? Or possibly SAFe (Scaled Agile Framework) which talks about program implementation through longer planning and execution known as Agile Release Train?

What is agile?

Agile when I say refers to Manifesto for Agile Software Development that was defined by 17 developers in February of 2001 and consists of 4 values and 12 principles. Many claim the Agile way of working was in existence before 2001 and I believe that’s true because Scrum was introduced through the HBR paper “New New Product Development Game” in 1986. I admit my ignorance of anything beyond that so I will stop at that. Now, however, we see many versions of agile like agile way of working, enterprise agile business agility, and so on.

What is Waterfall?

The waterfall is about executing work in phases or sequence as we do for a data center - plan, design, build, commission, optimize, assess, etc. and involves capacity planning, cost analysis, civil work, procurement, installation, commissioning certification, etc. These can’t run in parallel like what we do in software development where everything gets executed in parallel or overlapping. The first formal description of the waterfall model is often cited as a 1970 article by Winston W. Royce, although Royce did not use the term waterfall in that article. Royce presented this model as an example of a flawed, non-working model; which is how the term is generally used in writing about software development—to describe a critical view of a commonly used software development practice. The earliest use of the term "waterfall" may have been in a 1976 paper by Bell and Thayer. Source - Wikipedia

Is PMI - Project Management approach similar to waterfall?

Can be perceived that way if you read it incorrectly, but does it promote one-time planning and execution? The image below clearly promotes continuous planning and execution and talks about regular monitoring &controlling. It takes about the same PDCA (Plan-Do-Check-Act) cycle that we talked about in the name of agile development and states that initiation & closure is one time that too not applicable at the project level but at all phases.

So what is PMI promoting in the name of PMBOK?

The project management body of knowledge (PMBOK) talks about Knowledge Areas, Processes Tools & Techniques. This knowledge is really useful even in an agile context. It is about understanding in regard to requirement management, scope management, quality management, risk management, and stakeholder management. You will find plenty of tools & techniques to manage work. PMBOK talks about FIVE phases of the project namely

• Initiating – The initiating phase talks about the goal, purpose and objectives
• Planning – Planning everything that is needed to execute the project
• Executing – This phase is about doing actual work
• Monitoring & Controlling – This is about the periodic review of work and taking corrective action
• Closing – This is about lessons learned during the various phases.

What’s wrong with PMBOK?

If you talk in terms of the project then there is nothing wrong with PMBOK and fits perfectly fine for the predictive complex project. It defines a project as a temporary endeavor undertaken to create a unique product, service, or result. Here the expectation is requirements are clear (maybe complex but clear) and required complex technologies, processes, and tools to execute are identified and available. Usually, such projects take anywhere from 5 months to 5 years. Most project is about completing work and handing it over to the operations team to operate, maintain, and sustain.

The typical software product has some basic challenges that don’t fit within this cycle like frequent changing requirements for it is based on a feedback cycle thus requiring more frequent cycles (of what?)and doing all steps as stated by PMBOK may not be possible or required.

Also, the product development cycle is continuous rather than temporary and work keeps emerging as more is learned about customer expectations, changing business needs, and even the technology upgradation. Product development starts with an idea and will continue till the product is retired so planning and visualization of the whole life cycle upfront is not possible.

Even Scrum (the most popular Agile approach) does mention the project and clearly states that every cycle (Sprint) can be considered as a project.

What is there in Manifesto for Agile Software Development?

The manifesto’s primary focus is software development and consists of 4 values and 12 principles. I am not quoting those principles here as they are very specific to software development but the stated values are relevant to many other areas and can be easily interpreted by anyone who understands project or product development.

• Individuals and interactions over processes and tools - It doesn’t mean we don’t need processes and tools but it means the focus should be more on people and interactions between them. So, some of the PMBOK processes may fit in here.
• Working software over comprehensive documentation - Documentation is fine as long as it adds value. Just producing documents and not products shouldn’t be the focus.
• Customer collaboration over contract negotiation –When executing complex work, negotiating requirements, resources, and deliverables becomes complex and everyone tries to tilt the contracts in their favor. The agile approach however talks about building trust by the two parties collaborating rather than signing a fixed contract. This by no means indicates that we don’t need contracts and accountabilities.
• Responding to change over following a plan - We do need plan and planning for sure, but the Agile approach suggests planning frequently rather than once upfront so as to address the changing needs.

The whole manifesto promotes a people-centric approach over having rigid processes to address complex adaptive work.

So, what approach works well?

As I mentioned earlier it depends on the nature of work but let’s also explore if the two approaches really contradict each other or do they complement each other. My perspective on these two approaches is If you are talking about Project Management then the PMI approach suits you better but adopting agile values does make sense.

For instance, can the project team be more collaborative in approach to dealing with complex issues by simple conversation rather than going through stated processes? If yes then why not adopt these values there? Can we go further?

Yes, we can learn some good practices from the Scrum framework even if we do not have releasable increments at the end of Scrum and can’t produce. But we can still practice daily scrum, monthly review, and respective scrum values. If you are talking about complex product development then agile approaches such as Scrum, XP, LeSS, Lean, and other similar methods make a lot more sense. But even in those cases, knowledge areas, techniques, and tools from PMBOK such as requirement management, quality management, risk management, and stakeholder management do add value for these are knowledge not process and this knowledge is still required and useful.

What is the Agile Way of Working?

This has more to do with organizational structure, processes, and people within the organization to adopt the lean approach in order to respond to the fast-paced changes of the VUCA world. The Agile Way of Working is much more than agile vs waterfall and needs more time to understand and I promise to write about it some other day.