The global DevOps market is set to hit $9.4 billion by as early as 2023. Yet, 78% of the organizations have failed to get it right. Why?
DevOps celebrated its 10-year anniversary in 2018. In the technology industry, that’s a lifetime! Despite its relative maturity, the DevOps philosophy continues to evade even the most prominent and resourceful organizations. A shocking Gartner report has revealed that 75% of the DevOps projects fail to meet their goals.
Why are DevOps failure rates so high? What are the common challenges faced by organizations when implementing the DevOps philosophy? How to overcome these challenges?
We’ll tackle these questions and offer businesses replicable tactics to improve the success rates of their DevOps initiatives.
DevOps challenges can be broadly classified into People, Process, and Technology challenges. There’s no one set of organization-wide policies for businesses to adopt and guarantee project success. Every organization must develop a DevOps policy aligned with the organizational culture, business goals, and customer needs. However, here are some of the common challenges all businesses face, and how to overcome them.
1. Irregular Resource Allocation
Resource allocation is a major DevOps challenge. The mere integration of development and operations teams will not produce a high-functioning DevOps team. An alarming number of DevOps teams lack subject matter experts, which severely affects the team’s ability to deliver on DevOps’s promises.
For one, a generalist working on different technologies involved in app development, optimization, and monitoring, will not be as productive or efficient as an expert. It wastes precious time, ultimately slowing the DevOps delivery speed.
Also, DevOps teams work best when they minimize unplanned work. Without dedicated resources working on niche DevOps issues, DevOps teams are forced to assign complex issues to non-subject matter experts. This throws a wrench into their planned work schedule and makes the entire team inefficient. What’s more, the increasing workload on such talent will lead to employee burnout and runs the risk of derailing the entire DevOps initiative.
DevOps accelerates feature releases, updates, and time-to-market only when there is specialized talent working on problems. Therefore, businesses must identify key app technologies and development processes that can be streamlined with DevOps, and assign dedicated, skilled talent to those specific areas.
Optimum resource allocation is critical for the success of DevOps initiatives.
“Executives who don’t realize that DevOps is a critical foundation to any IT transformation business should not be in the business of leading IT.” Anaf Durrani, VP Engineering, Grainger.
2. Misalignment of Responsibilities
DevOps brings teams of vastly different goals together to work in a ‘volatile’ environment. Developers are primarily concerned with innovation, operations teams with stability, QA teams with flawless performance, and so on. Naturally, there’s bound to be friction and conflict between these teams.
Making matters worse, the higher management doesn’t always define the DevOps team’s goals, responsibilities, and priorities clearly. This leaves a great deal of room for ambiguity. Teams who are used to working in silos without worrying about dependencies go back to their old ways, thereby negating all efforts towards seamless collaboration.
Getting teams out of their silo mindset is the biggest challenge before change leaders. For this reason, DevOps works best when teams are made of interdisciplinary resources. Developers with an operations mindset, who don’t shy from going out of their comfort zone routinely, are direly needed for spearheading DevOps initiatives to success.
Organizations typically overcome these challenges by clearly delineating DevOps goals, priorities, and responsibilities. More importantly, they assign integrated responsibility for the success of DevOps tasks. Each team member is responsible for the success of DevOps tasks end-to-end. When their individual performance is measured by the team’s overall success, silos automatically break down, and collaboration increases rapidly.
3. Fragmented Processes
Not many DevOps leaders realize, or at least appreciate, that the DevOps is deeply fragmented. Despite its maturity, DevOps is not the choice of software development and delivery model for SMEs. DevOps has been primarily a large enterprise initiative for a long time. For this reason, SMEs that take on DevOps find themselves in the deep end of the water.
DevOps works by automating the bulk of the tasks involved in the software development life cycle. However, there’s no one tool, process, or resource to make that happen. DevOps teams must use different tools for automating different aspects of their operations. There are great tools for automating individual components like continuous integration, infrastructure provisioning, testing, source control, and so on. However, these tools do not talk to each other.
Making these disparate tools talk to each other requires massive resources that most organizations cannot or are willing to allocate. For this reason, DevOps teams are often forced to work with limited automation capabilities, which is the antithesis of DevOps.
Effective DevOps teams divide their time between performing their tasks and automating those tasks. Without the latter, the former increases progressively, resulting in employee burnout, delayed processes, worsening responsiveness, and a drop in quality of deliveries.
Businesses can avoid these problems by developing a crystal-clear DevOps strategy that specifies the organization’s DevOps goals, identifies the processes that can be automated, and deploys resources to realize them. The goals should match the resource allocation, and this realistic approach to addressing defragmentation will help businesses streamline and automate processes that matter to them.
4. Lack of Proper Metrics
Lack of proper metrics is as much a process challenge as it is a people challenge. KPIs and metrics go a long way in conveying DevOps teams the organization’s priorities and expectations. As discussed before, stability and deployment time continue to be in perpetual conflict within the DevOps teams. Should they rush deliveries at the cost of stability or vice versa? How do they even begin to prioritize one goal over another?
Metrics offer unambiguous and precise directions for teams to prioritize different goals. Although the value of these metrics may change from one business to another, the metrics themselves are universally relevant for all DevOps teams. Here are some metrics that businesses must define when communicating DevOps goals to their teams:
- Deployment frequency.
- Deployment time.
- Change failure rate.
- Automated Tests Pass Percentage.
- Number of errors following each delivery.
- Defects escape rate.
- Time to detection.
- Feature usage.
- End-user experience.
- Business impact.
- Failed deployment.
5. Cultural Changes
Resistance to change will be the biggest roadblock to DevOps transformation. DevOps seeks to shift control from siloed teams and their heads to a single multi-departmental outfit within the organization. Naturally, such attempts can be interpreted as an erosion of decision-making power.
Further, it’s not all about control. DevOps leadership has vastly different roles to play compared to traditional IT roles. Generally, IT leadership must possess the expertise to guide, support, and advise their employees on various technologies. In a DevOps environment, that’s not the case. DevOps employees are working in a volatile and rapidly evolving environment. Mistakes are common, and the consequences of those mistakes can be disastrous. It’s not difficult to see why employees may be apprehensive of the DevOps process.
Therefore, the primary role of leadership is to create nurturing conditions, where the employees are accorded a higher degree of freedom and are protected from setbacks that come with rapid experimentation. Additionally, the leader’s efforts must be focused on identifying DevOps success patterns and replicating them to scale up the transformation across the organization.
A top-down approach that seeks to redefine the role of leadership empowers DevOps teams with more freedom for experimentation and assures them of stability is critical for overcoming the cultural inertia.
“Inability to Enforce a Cultural Change — The stakeholders throughout an organization must all be in alignment to support the necessary cultural changes required for a successful DevOps adoption. It includes the executives and leaders within the various groups. It is not just a technical adoption. The business, operations, IT, finance and others must commit and establish trust in order to succeed.” Ian Willoughby, VP Cloud Solutions and Chief Architect, 2nd Watch.
6. Inability to Scale Up DevOps
More often than not, the very success of the early DevOps initiatives becomes their failure. The top-performing DevOps teams are inundated with more projects, and they promptly end up becoming the bottleneck for project delivery. Not to mention the increased stress and dropping productivity that comes with it.
An obvious solution to this problem would be to expand the DevOps team. However, that’s easier said than done. DevOps specialists need a vastly different skill set than developers or engineers, and therefore, are difficult and more expensive to hire.
Some organizations overcome this challenge by embedding a DevOps specialist in every development team. Their role would be to streamline the respective team’s delivery chain while coordinating with other departments’ DevOps specialists. However, this approach often breeds inconsistencies and collaboration hassles between teams. One way to address these new problems would be with an inner-source methodology, which draws from open-source learnings.
Teams must be empowered with powerful collaboration tools that allow them to code, ship, and collaborate from anywhere in the world. Lastly, a robust decentralized ‘product-focused’ delivery model must replace the traditional ‘project-centric’ approach.
“Changes driven by DevOps first require a purpose in which to believe. Then, to scale the changes successfully requires communication, collaboration and commitment throughout the organization.” Qasim Khan, SVP — Business Information Officer, U.S. Bank.
7. Inability to Incorporate Security
On a purely superficial level, DevOps and security appear to be in total conflict. DevOps is all about speed and continuous delivery, while security emphasizes extensive testing and fool-proofing. However, organizations are slowly realizing that DevOps integrated with security can help them patch vulnerabilities, release updates, and respond to cyber threats faster than ever.
At present, DevOps faces three hurdles when integrating security into its processes — Slow Pace of Development, Exhaustive Security Standards, and Visibility into Threats.
Thankfully, there are ways to address these challenges. Security automation tools are compatible with DevOps workflows and can help drastically improve security reviews, vulnerability detection, and patching. The same tools can be used to automate security standards across the development. Of course, that would require organizations to drop the bulk of their security standards and retain only the core security standards for DevOps.
Lastly, threat visibility can be improved by making security data available to all team members and reporting easy for them. A SIEM dashboard that’s customized to each team member based on their roles and responsibilities can go a long way in giving threat visibility to DevOps teams. To make it make effective, a rewards system based on shared performance goals can be set up.
“The value proposition for security is synonymous with the DevOps objective — to reliably build, test, and release software faster. We have the ability to do address security during development, but it is our follow-thru to deliver that sets organizations apart. People and Process are the most important considerations — training teams to address security as part of the development process.” Joe Burkard, Security Advisor and former CISO, Protiviti.
Every organization’s DevOps initiative will be met with complex hurdles that are unique to that organization. However, a strong focus on collaboration and assurance of stability to team members can reduce internal resistance and turn potential sources of inertia into change-leaders, thereby ensuring success.
