It appears that the long-standing dichotomy between Waterfall and Agile project management approaches is coming to a close. At the core, these two methodologies differ in their respective approaches to predictability and adaptability. Waterfall projects are characterised by the need for all predetermined features to be completed and implemented before the project can be considered finished, while Agile projects are geared towards the release of a minimum viable product, which is then improved upon through user feedback that helps to determine the ideal direction for further development.
What Exactly Is Agile-Waterfall Hybrid?
Since its official introduction more than two decades ago, the Agile methodology has seen a surge in popularity and has become increasingly prevalent within software development areas that had previously relied heavily on Waterfall methodologies. To capitalise on the advantages of both approaches, a hybrid approach which formalises the use of Agile within the broader Waterfall structure has been developed.
Jim, a Works project manager, Scrum master, Agile coach, and instructor in the Boston area, remarked that “true hybrid work is a combination of predictive and uncertain work.” He explained that while Waterfall techniques are used in a hybrid model for the more predictable components of the project, Agile techniques are adopted for the parts that are more iterative and less certain.
By leveraging Agile methodology during the software development process and Waterfall methodology during the deployment process, companies can maximise the efficiency of their product development. For example, a financial firm may have to go through a rigorous auditing process for larger components of their product, which requires them to adhere to the Waterfall model of deployment. However, the development team can still utilise Agile practices when iterating through smaller pieces of the feature, as well as other user interface elements, during sprints.
Flexibility is of paramount importance when it comes to devising the ideal hybrid system for any given project. Mike, a member of the Project Management screening team at Works and a contributor to this blog, is based in Belgrade, Serbia, and he has a unique outlook when it comes to this issue. “I am firmly of the opinion that one-size-fits-all is not the way to go,” he says. “It is simply not enough to just look up a few methodologies in a book and apply them indiscriminately. You must comprehend how to adjust these processes according to different elements, such as the maturity of the team, the culture of the organisation, the culture of the particular subject matter, the type of project, the size of the team, and the size of the product.
Why Should You Use a Hybrid System?
It may be tempting to consider transitioning to an Agile methodology, given its reputation for being more adaptive, creative, and effective. However, for an organisation that is used to working within a Waterfall system, it may not be the most appropriate choice.
John, a Córdoba, Argentina-based project manager and member of the Works’ Project Management screening team, highlights the difficulty of implementing Agile in highly regulated industries. This is because products in these industries are subject to strict approval processes and timetables, which require documentation and auditing. In such cases, where risk is a primary factor, the Waterfall method may be more suitable. John brings up his experience in a clinical trials company, where audits are particularly stringent, and external authorities such as the FDA must be accounted for. He explains that the Agile approach, which involves adjusting or reprioritizing scope and backlogs, may interfere with these audits.
According to Dave, a Works project manager based in Brussels, Belgium, careful consideration must be taken when considering the timing of releases for jobs that require privacy protections. Dave recently led a hybrid team in the development of a Microsoft Azure Active Directory identity management platform for a European banking group. In order to ensure the bank’s privacy data was adequately protected, the team decided to use Agile for some development steps, but provisioned the system on a local server. As Dave explains, “First, the on-premise implementation must be completed, followed by the cloud implementation. After these steps have been completed, the link between the two can be made. It is crucial that these steps are completed in the correct sequence for the project to be successful.”
Due to the combination of stringent regulations and the necessity to protect data, the majority of project managers that we consulted for this article have all worked on hybrid projects for clients in the financial sector. Grant, a Works project manager based in Johannesburg, South Asia, has considerable experience in financial services, banking, and stock exchanges. He noted that he has successfully implemented an equities trading engine and a derivatives trading engine, both of which contained a high degree of integration in relation to master data as well as a considerable amount of regulatory requirements.
The implementation of a Waterfall approach is often necessary to ensure that the necessary steps are being followed, however, Agile principles can be incorporated to improve the process. Breaking down large epics into smaller user stories allows for greater flexibility in the development process, but the epics can still be released over a larger period of time. As Schuleman mentions, this can be done via a Big Bang delivery, where the team deploys incrementally to a user acceptance testing environment. Once all the features have been approved in the UAT environment, the project can then be released to production, which may take up to a year depending on the scope of the project.
Schuleman’s most extensive program involved 120 personnel working on 10 projects, which included a mix of Waterfall, Scrum, and hybrid approaches. To ensure that all of the smaller teams were appropriately coordinated for the upcoming sprints and operating at harmonious speeds, Schuleman convened a “Scrum of Scrums” every two weeks.
The Most Difficult Aspect of Going Hybrid
As the project manager, it is essential to identify the most appropriate combination of Agile and Waterfall methodologies for the product, team, and people involved in order to achieve successful implementation of an Agile-Waterfall hybrid system. According to Mike, it is likely to be unsuccessful if these methodologies are applied without any customization to fit the specific requirements of the project.
Grant attempted to introduce Agile processes into a modernised version of an existing application, but was ultimately forced to give up on the project. The experiment failed because the developers, all of whom had been accustomed to using Waterfall methodologies, were confused by the division of tasks into epics and user stories. Grant observed that, when presented with a user story assigned to a sprint, the developers would question why “there are 10 other user stories also related to this application that are not yet in the scope.” It was clear that they wanted to work on all aspects of the project simultaneously.
It is essential for the success of a hybrid system that the members of the population are open to change and willing to embrace new ideas. According to Jim, many team members and management personnel lack a sufficient understanding of project management, and are more likely to stick to traditional methods such as the Waterfall model due to their lack of familiarity with the new process. Therefore, it is important to ensure that the population has an understanding of the new hybrid system, in order for it to be adopted and implemented successfully.
In addition, utilising a hybrid methodology is not right for every situation. According to some project managers, combining different methodologies may create more complications than it resolves. John commented, “Generally speaking, utilising a hybrid approach is not a wise decision. You’re increasing your odds of failure by combining the worst aspects of both worlds.” Agile processes are designed to be flexible and open to change, but this is limited when used in a Waterfall environment. The linear and fixed nature of Waterfall does not allow for changes to be easily made and can be expensive. By incorporating Agile into a Waterfall structure, you can introduce nonlinear elements, but this can come with its own set of challenges.
Despite the potential risks associated with hybrid projects, when managed and implemented correctly, they can prove to be highly beneficial. Mike recently had the opportunity to lead a successful hybrid project in Republika Srpska, Bosnia and Herzegovina, for the International Finance Corporation (IFC), a member of the World Bank Group. According to Mike, it was a plan-driven project, but both parties agreed to use a mix of traditional and Agile approaches. All requirements, specifications, and other necessary elements were thoroughly detailed, but the teams still worked in an Agile framework.
The Republika Srpska government was able to reap the benefits of this process, as they were able to acquire a higher quality product at a more affordable price and within a shorter duration of completion. According to Mike, many different organisations were involved in the process, ranging from governmental organisations to other unrelated entities. Nevertheless, despite the complexity of the process, it was ultimately successful.
Hybrid Technology as an Agile Upgrade
In recent times, as digitalization has become increasingly prevalent across various industries, there has been a growing trend of businesses showing increased interest in the Agile methodology. Jim has commented that Human Resources departments are quickly embracing Agile, and there is even a police force in the Netherlands that has successfully utilised task boards to reduce their backlog of criminal cases.
Although your business may not be immediately prepared to fully embrace Agile principles, the long-term rewards of gradually introducing Agile methodology can be significant. Chief among the benefits of Agile is its ability to effectively navigate uncertainty. As noted by Schuleman, “Agile is far more advantageous in circumstances of uncertainty, which are in fact the norm rather than the exception.”
One of the key benefits of Agile is its transparency. According to one user, “I prefer Agile because I can easily observe the progress being made. With a Waterfall approach, there is a tendency to make exaggerated claims about the progress being made, such as ‘We’re 20% complete,’ or ‘We’re 30% complete.’ However, it is common for progress to plateau at 80% for an extended period of time, making it easy to conceal any difficulties. With Agile, daily standup meetings ensure that if any user stories remain stagnant, it is easy to identify the issue and take corrective action.
Jim believes that a significant amount of IT projects fail each year due to a lack of Agile methodology. He believes that, had Agile been incorporated into the project, it could have saved both money and time while providing more value to the customer. He suggests that the 35% to 40% of IT projects that fail annually could have been prevented if Agile had been implemented.
Mike believes that introducing a hybrid system to gradually incorporate Agile practices can be beneficial, particularly when certain Waterfall approaches can be advantageous. This blended system focuses on the discovery phase, creating a more detailed product backlog than would be used for a pure Agile project. Additionally, this approach helps the team gain a more comprehensive view of the project’s end goal. Mike emphasises that the entire team should understand the product details in order to develop a sense of product ownership.
Your organisation may not be in a position to fully embrace an Agile methodology, but you can still take advantage of some of its key principles to improve project performance. Strategies such as daily standups and shorter, more regular delivery cycles can yield considerable benefits without the need for a complete transformation. However, for a hybrid approach to be successful, you must be diligent, wise, and cautious in its implementation.