Scaling DevOps to Enterprise? Here are the pitfalls to avoid.
DevOps as a model of software product delivery is becoming increasingly prominent in larger enterprises. It is a system of work that, amongst other things, facilitates higher and faster rates of change, through the optimization of development and delivery processes. At its core, DevOps is a working culture that dissolves the traditional silos between development, operations, and other stakeholders. DevOps has the capacity to improve business outcomes, and that too, rapidly. DevOps is characterised by agility, lean practices, open-source software, transparency, and collaboration.
DevOps Enables Change
Businesses are increasingly adopting a DevOps approach as a way of coping with the fast pace of change and seemingly never-ending wave of disruptions. Regardless of the sector, disruption is guaranteed. From financial services, healthcare and hospitality to education and government, organisations that intend to continue delivering services that meet customer needs must adopt fleet-footed practices to remain relevant and profitable. One of the core reasons many enterprises remain vulnerable to disruption is the slow-pace of legacy operations, and traditional siloed organisational structures.
DevOps has the potential to change this. Contrary to popular belief – DevOps is not a technology play made up of a series of tools and products. It is a business play. The benefits far outreach the realms of Software Development or IT Operations functions.
DevOps has more or less emerged from the Agile philosophy and is a pragmatic example of how agile principles can deliver actual business results. Typically, DevOps aims to reduce time-to-market and increases the quality of software applications, thereby enabling companies to respond more proactively to change and release.
DevOps is for Business
DevOps is not just about outputs. It is a state of readiness throughout the app production sequence; training developers to respond with transparency and speed, while maintaining high degrees of collaboration and communication. DevOps holds the key to optimising an enterprise’s entire operational and organisational structure. Again, beyond a technology play – DevOps is a business play.
Enterprise DevOps is about taking the lean processes, typically being used in web companies, to dramatically increase speed to value and applying it to an Enterprise.
We aim to evaluate people, process, and technology across the enterprise to create a staged roadmap based on a targeted level of maturity. Read more about our approach to Enterprise DevOps here.
Scaling DevOps across Multiple Departments
When DevOps is deployed in software and application delivery, very often the range of stakeholders is necessarily broadened to include security teams, database administrators and line-of-business leaders. Furthermore, results are even better when DevOps teams are populated with members from other functions such as business, fiscal or legal. Therein lies an extraordinary opportunity to create rapid response teams, with a variety of skills, and which can mobilise products, apps, and services directly to customers.
Ironically, as fast and nimble as DevOps is, scaling it, is not. DevOps is not an off-the-shelf solution, nor is it something that can be rolled out by consultants. It is a fundamental operational, as well as organisational change. As with all change, DevOps too comes with some pain.
But if you have already embraced DevOps or you are on your way there, it is imperative that you examine ways to avoid a few painful pitfalls when it comes to scaling your DevOps approach enterprise-wide.
DevOps Culture and Perceptions
You may already have small, multi-disciplinary DevOps teams delivering on isolated projects. Your DevOps “experiment” may have yielded great results, which gained the attention of the C-suite and other managers. Soon your small team is inundated with request and projects, and a broad decision is taken to employ DevOps across far more development cycles.
I encourage you at this point to take one step back. One swallow does not a summer make, and initial enthusiasm for DevOps is unlikely to melt away generationally entrenched hierarchies and systems of control upon which most enterprises are built.
The ING DevOps Experience
Dutch banking group, ING, had an interesting journey towards adopting agility across its cultural, organisational and infrastructural models. Although we are discussing the more precise application of a DevOps approach, ING’s road to Agility is most relevant. With an ambition to design services that were more customer-centric, resistant to disruption and more swiftly delivered, the bank looked to tech unicorns such as Spotify and Netflix for inspiration. It was agreed that ING needed to start acting more like a tech company, and less like a bank. Understandably, one of the most pertinent challenges was a “cultural shift”. Banks and financial services are not typically built for agility, but instead stability, predictability, and security. While this story has a happy ending: ING now consists of 350 nine-person “squads” in 13 so-called tribes, that deliver better services and solutions to customers quicker. The lessons learned in the three-year journey are also excellent markers of how to approach scaling DevOps:
Have a plan: Develop a business-oriented rationale for adopting DevOps
Go Slowly: Without reverting to siloed thinking, one should adopt DevOps strategies one value stream, or one project, at a time
Look beyond your own sector: be willing to learn from others
The first lesson learned by ING, as above, is also the most important: It is imperative that an enterprise-wide DevOps approach be founded on actual business objectives. Ideally, a DevOps strategic document that defines the goals and outcomes from an operational and market perspectives, should be authored and issued by the C-Suite.
“Without a prescriptive path forward, organizations are struggling to scale their DevOps success beyond isolated teams.”
State of DevOps 2018 Report
DevOps - Generalisation vs Specialisation
The truth is that large organisations don’t attract or require generalists, they require specialists. And specialists prefer silos, cocoons, and firewalls. As attractive as the outcomes of DevOps are, there will not be universal buy-in, and certainly not from functions such as IT infrastructure, database administrators or security specialists. Your origination document, and its attendant vision for the company, rationalises what will for many be a tricky change. Embroiled in discussions, planning sessions and inevitable debates, a DevOps strategic document can be the touchstone that keeps the discussions on track and moving in the right direction.
Leading the Enterprise DevOps function
The fluidity and speed of DevOps will not come naturally for the IT operations professional, who is used to focusing on infrastructure stability and security over a period. Therefore, leadership is critical: a top-down vision and message will help bridge the dev and ops divide; ideally your C-suite should become DevOps ambassadors.
It boils down to managing, and mostly changing, the metrics by which the entire process is measured. One of the principles that apply is “fail fast” – the idea that if it doesn’t work, rather we know sooner than later. Most enterprises do not count failures as successes, and have developed an inner self-measure that fears failure. Because DevOps flattens and compacts the development life-cycle, testing happens more rapidly. Full-system deployments – when using concepts such as containerisation – can also be executed quickly. In this way failures and errors are swiftly identified. This is a good thing. And so, perceptions of failure must change.
Measure Your Success
While DevOps may be fluid, and to some degree organic in its approach to delivery, the deliverables themselves must be accurately measured. As with any good development cycle, the quality of the feedback defines the quality of the next iteration. It is the same with DevOps deployment. Retaining C-suite level enthusiasm for the DevOps project is also easier when outcomes can be measured, preferably in dollars and cents.
Businesses tend to measure their outcomes in a number of standard ways: improve customer experience and satisfaction, create better applications and services, produce cost savings, generate profit, reduced time to market and increase efficiencies. DevOps metrics are framed differently. Puppet’s recommended DevOps metrics include: Deployment Frequency, Change Lead Time, Change Failure Rate and Mean Time To Recover (MTTR). Critically for DevOps team leaders and CIOs keen on maintaining momentum for the DevOps roll-out, they will need to translate DevOps metrics into hard business outcomes.
Developing a model that works in small teams, then extracting from that model formal practices and standards is a manageable way to start scaling your DevOps enterprise-wide. Once team leaders have worked out “the recipe” for how DevOps has so far succeeded in the enterprise, they can then develop a repeatable process and workflow across more teams, that is more likely to succeed. Keeping in mind the imperative of metrics as described above, the success we describe here, must be a quantifiable and demonstrable business-based outcome. Ensure also that part of your “recipe” is a set of metrics that define what the desired business-outcome for that team is. These KPIs will be signposts and goals which guide and encourage DevOps teams towards greater alignment with the goals and objectives of other enterprise functions such as marketing, legal, finance, and compliance.
Measures and Metrics for Enterprise DevOps
Again, the transition from metrics that measure development and deployment, to metrics that measure business outcomes should begin at a small scale – it is far easier said than done. Most crucial is the ability to adapt to the metrics. You may find that your team’s frequency of deployment metric is encouragingly high, but that a correlated increase in customer satisfaction is negligible. It can be difficult to have a well-oiled DevOps team that is nonetheless not hitting business targets. However, as with most agile systems, changes can be rapidly made and fast fixes deployed to address the lack of business outcomes head-on. In these situations, transparency and trust are essential. Upper levels of management must have enough trust in the intent and process of the DevOps roll-out to allow for the mistakes and the fixes.
DevOps-at-scale requires a permissive and flexible change culture. This trust will also yield the support, resources and time needed to test how smaller DevOps teams deliver on business outcomes, resulting in a better DevOps model that can be more readily replicated elsewhere.
Having the right tools is a major driver of success with DevOps. Automation tools mean companies can quickly pivot in response to change or risk. Agility demands high levels of feedback, which in turn requires rapid testing cycles. While good DevOps systems rely on automation, they have also isolated which development processes can and should be automated, and which should remain manual. When challenged with scaling up a project-based DevOps approach to something more all-encompassing, the natural inclination is to “automate more”. However, jumping feet-first into large-scale automation can create problems when scaling your DevOps. The secret to successfully scaling DevOps to a system of continuous delivery, is understanding that your business, development cycle and resources are unique. As such, and as with most DevOps good practice, one needs to start at the granular. Automation should be as carefully selected and attributed at scale, as it is in smaller scenarios.
Rather than being tempted to enforce the same set of tools that worked in smaller teams, on the entire organization, allow for more flux. Driven by their stated business KPIs and outcomes, let your new DevOps teams adopt a connected yet loosely-coupled DevOps toolset, should they require it. These days most DevOps tools can function symbiotically, and despite being from different vendors can nonetheless maintain the necessary end-to-end communication and collaboration channels required.
In Conclusion: Enterprise DevOps is more than Tech
While smaller DevOps nodes may have developed organically, alongside great success, there is no easy way to replicate that success across the enterprise without having a few essential processes and principles in place. The most critical is a plan: a compelling business case for the change, alongside a mapped-out vision for how that change will be implemented and at what pace. Secondly, everyone involved in the DevOps roll-out should know what success looks like: which metrics will be used to measure the roll-out, and what business goals need to be achieved. This clarity creates useful alignments between previous silos, and will also give teams hard targets to aim for. Thirdly, start small. Find out what works in smaller teams, extract the formula, and look to repeat that standard across other teams, incrementally. Finally, there is no need to be mesmerised by the plethora of DevOps tools available. Keep in mind that DevOps, in its essence, is not something you can buy, it is something you need to do. And while there are tools that can and should aid you on your DevOps journey, the key remains broad-based buy-in from your teams and the trust of your C-suite and board.
LimePoint is an experienced specialist in Automation, DevOps, and Continuous Delivery. Our mission is to empower the enterprise in the cloud economy. We do this by providing a suite of automation powered products and services. Our automation and configuration management products enable organisations to deliver automation capabilities across the entire platform and cloud lifecycle - from provisioning, through to management, and finally retirement of their systems and software assets, regardless of the underlying platform. Our suite consists of two products, MintPress and DriftGuard.
To learn how LimePoint products and services can help your organisation, please contact us.