← Back to Blog RSS

DevOps best practices: what works, what scales, and what breaks

DevOps CI/CD Infrastructure

DevOps best practices are well-documented, but this guide goes beyond the point where those practices start to create new problems, specifically covering how infrastructure state is managed as teams and environments grow.

TL;DR
$ cat devops-best-practices.tldr
• DevOps brings together development and operations disciplines to shorten release cycles and improve reliability.
• The core DevOps practices (version control, continuous integration or CI, continuous deployment or CD, automated testing, IaC, and monitoring) form the foundation most teams build on.
• These practices are increasingly standard, the differentiator now is how well they hold together at scale.
• IaC and state management are where many teams hit a ceiling: the practices are sound, but the underlying tooling was never designed for the concurrency and complexity of modern infrastructure.

DevOps brings development and operations teams into the same operating model, with shorter release cycles, tighter feedback loops, and shared responsibility for reliability.

The core DevOps practices are no longer controversial. Version control, continuous integration, continuous delivery, continuous deployment, automated testing, infrastructure as code, and monitoring and observability form the foundation most software teams are trying to build on.

Adoption is not the differentiator anymore. Redgate's 2026 State of the Database Landscape survey found that 64% of organizations are struggling to apply consistent practices across environments as estates become more complex, even as fewer organizations report managing more than five databases.

Bar chart from Redgate's State of the Database Landscape survey showing how many database platforms organizations use across 2023, 2024, and 2025, with the more than five category falling from 29% to 8%.

The pressure has shifted from sheer database count to the harder problem of keeping practices consistent across environments, teams, platforms, and workflows.

The ceiling usually appears where CI/CD and infrastructure as code meet shared state. Your DevOps best practices are sound, but file-based Terraform state, global locking, drift detection, and weak infrastructure-layer visibility create coordination problems that your original workflow was never designed to absorb.

What DevOps actually means

DevOps is a cultural and technical approach that brings together software development and IT operations so teams can shorten the software development lifecycle, improve reliability, and remove the hand-off friction that used to sit between people writing application code and people running production systems.

It emerged as a reaction to slow, siloed release cycles, where a development team would finish code changes and pass them over to operations teams as if deployment, infrastructure provisioning, monitoring, and incident response were someone else's problem.

That model could work when releases were rare, environments were static, and production changed slowly, but it breaks down once software delivery becomes a continuous process.

Today, DevOps means:

The 7 core DevOps practices

The DevOps lifecycle is commonly described through seven phases, with each practice creating a tight loop between intent, change, verification, deployment, and feedback.

Version control

The foundation is version control, which gives the organization a single source of truth. Application code, infrastructure configuration management, pipeline definitions, test suites, alert rules, runbooks, and policy definitions should live in a version control system rather than on laptops, in consoles, or in undocumented automation tools.

Without version control, there is no trustworthy audit trail, no reliable rollback path, no clear ownership of change, and no repeatable way to reproduce an environment. With it, DevOps teams can review changes before they land, understand why something changed, and trace production behavior back to a specific commit.

Continuous integration

Continuous integration is the term used for when developers frequently merge code to a shared branch, with every pull request triggering automated builds, unit tests, integration tests, and whatever smoke testing or static analysis the team has decided is necessary for code quality.

Integration issues caught on the same day are small, specific, and usually easy to fix, while integration issues found after a long development cycle become archaeology. CI turns software development processes into a habit of constant reconciliation, where the main branch is always close to release because every change has already been forced to prove itself.

Continuous delivery

With continuous delivery, every change that passes automated checks is in a deployable state. It does not mean every change is automatically deployed to production, however.

The pipeline has built, tested, packaged, and prepared the change so the deployment process is routine. The change actually goes live.

Regulated environments, high-risk changes, and organizations with strict release windows may still keep a manual approval gate before production, but the pipeline before that gate should still be fully automated.

Continuous deployment

Continuous deployment removes the final manual gate. Code is merged, tested, built, and pushed to production without someone pressing a release button.

Manual approvals still make sense if you have certain security concerns, need to make specific infrastructure changes, or own high-risk production systems, but they can become a bottleneck.

Automated testing

Unit tests validate isolated behavior, integration tests prove that services and dependencies work together, end-to-end testing checks critical user flows, load testing exposes performance limits, and automation tests turn quality from a late-stage inspection into a continuous feedback mechanism.

Automated testing cannot be bolted onto the end of a DevOps pipeline and still do its job. It has to be embedded into CI, tied to pull requests, and treated as part of the definition of done. Otherwise, agile teams can move quickly through development and still discover too late that the same pipeline is carrying broken assumptions into production.

Infrastructure as code

Infrastructure as code (IaC) refers to when infrastructure is defined in code, stored in version control, reviewed through the same workflow as application changes, and applied consistently across dev, staging, and production environments.

It is the practice that turned cloud infrastructure from a pile of console clicks into something software teams could understand.

Treating infrastructure configurations as code gives teams repeatability, reviewability, and a way to make infrastructure provisioning part of the DevOps pipeline rather than a separate operations ritual.

Monitoring and observability

Monitoring tells you something is wrong. Observability helps you understand why.

A mature DevOps methodology needs both. Monitoring systems, alert rules, system metrics, logs, traces, deployment events, and infrastructure state changes are the feedback loop that closes the DevOps lifecycle.

DORA's software delivery performance framework has evolved from the original four key metrics into a five-metric model, but the standard benchmark most teams still recognize starts with deployment frequency, lead time for changes, change failure rate, and recovery time, with DORA now using failed deployment recovery time rather than the older MTTR framing and adding deployment rework rate as an instability metric.

A DevOps best practices checklist

A DevOps best practices checklist is useful only when it moves beyond naming practices and starts testing the operational layer underneath them. Most teams can say they use CI/CD, IaC, automated testing, and monitoring. The harder question is whether those DevOps processes still hold under pressure.

How DevOps practices are best orchestrated

DevOps practices work best when they operate as a system. Version control feeds CI, CI depends on automated testing, delivery depends on reliable packaging and deployment pipelines, deployment depends on observability and rollback paths, and the whole system depends on development and operations teams sharing ownership rather than transferring responsibility at arbitrary boundaries.

The first pillar is culture and shared ownership.

A development team that writes code and throws it over the wall has not adopted DevOps, even if it uses modern DevOps tools. Operations teams that own reliability but cannot influence architecture, testing, or release software are left carrying risk without agency. Shared goals matter because the software lifecycle is shared, whether the org chart admits it or not.

The second pillar is automation as the default.

Manual steps in a DevOps pipeline are not automatically safer. Sometimes they are necessary, especially where approval has real governance value, but every manual gate should have a reason beyond habit. Manual infrastructure provisioning, manual smoke testing, manual deployment notes, and manual security review all create opportunities for delay, inconsistency, and drift.

The third pillar is observability across the full pipeline.

Application-layer metrics are necessary, but they are not enough. Teams also need visibility into infrastructure state, deployment history, configuration drift, ownership, and the change history underneath cloud infrastructure. Tooling choices matter, but tools alone are not orchestration. The workflow has to be understood before the organization can pick the right tools to support it.

The ceiling appears where practices meet state

The core DevOps practices solve real problems. They shorten feedback loops, improve software quality, reduce release friction, and give teams a repeatable way to deliver software. The issue is that, at scale, the same practices create new coordination problems.

CI/CD increases the number of actors trying to change infrastructure. IaC increases the amount of infrastructure that those actors can touch. Drift detection adds automated jobs that need to inspect or compare live systems. Platform teams add shared modules and reusable environments. Security teams add policy checks.

Suddenly, the state layer is no longer being touched by one engineer running Terraform from a laptop. It is being touched by engineers, automated pipelines, scheduled drift jobs, and policy systems, often at the same time.

That is where many DevOps best practices start to bend.

State contention in CI/CD pipelines

Terraform state is shared mutable state, and shared mutable state needs coordination.

Traditional Terraform locking protects state by placing a single lock around the whole state file, which prevents corruption, but also means unrelated changes serialize behind one another. One lock guards the whole state, even when a change touches only one resource, and the probability of contention rises with team size, resource count, and change frequency.

This is what happens when file-based state is used for concurrent access patterns. A single global mutex can keep the file safe, but it cannot distinguish between two changes that conflict and two changes that merely happen at the same time.

Configuration drift undermines IaC

IaC defines the desired state, but production runs in actual state. Drift is the gap between those two realities.

Manual console changes, incident patches that never become pull requests, automated processes outside the IaC pipeline, managed service defaults, and partial rollouts all push environments away from what the code says they should be.

Configuration drift is the divergence between desired state and actual state. IaC helps because it makes desired state explicit, reviewable, version-controlled, and reproducible, but does not eliminate drift by itself.

The important part is incentives. If the fastest way to fix production is still a console change, drift will continue. If drift detection requires a global state lock that blocks the same pipeline teams are trying to accelerate, then teams will run it less often, ignore it longer, or route around it entirely.

Infrastructure-layer visibility lags behind application observability

Application observability has matured. Teams can usually see error rates, latency, traffic, logs, traces, saturation, and deployment events.

Infrastructure-layer observability is thinner. Teams can see that something exists, but not always what state it is in, what changed it, who changed it, which module owns it, what depends on it, or whether it has diverged from code.

That weakness shows up in audits, compliance reviews, cost analysis, and incident response.

Without queryable infrastructure state, compliance checks become manual sampling exercises, audits become spreadsheet archaeology, and incident investigation takes longer than it should because the infrastructure layer is not treated as a first-class observable system.

Stategraph and infrastructure state management

We built Stategraph for the part of the DevOps workflow that usually gets treated as plumbing until it starts blocking everyone.

Terraform and OpenTofu gave teams a way to define infrastructure in code, review it, apply it consistently, and bring infrastructure provisioning into the same development process as application code, but the state layer underneath that workflow still behaves like a shared file that has to be protected from concurrent access.

That becomes a problem when multiple engineers, CI/CD pipelines, drift detection jobs, and policy checks all need to interact with infrastructure state as part of the same continuous process.

Stategraph starts with resource-level locking because global file locking was never the right concurrency model for modern infrastructure workflows. When independent resource subgraphs do not conflict, they should not be forced through the same state-file lock just because they live in the same backend.

Stategraph uses graph-aware execution, resource-level locks, multi-state transactions, and concurrent execution so teams can apply infrastructure changes in parallel without treating every run as if it touches the whole world.

We also model state as a graph because infrastructure is already a graph.

Comparison of file-based Terraform state, where every operation reads, locks, and rewrites the whole file, against Stategraph, which models infrastructure as a graph of queryable resources so a change touches only what it affects.

The problem is that traditional state backends flatten that graph into a file, then ask teams to infer relationships, ownership, drift, inventory, and blast radius from something that was never designed to answer those questions.

With Stategraph, infrastructure inventory becomes queryable. Teams can inspect resources across states, search and browse their estate, query infrastructure with a SQL-like language, build dashboards, identify gaps, visualize dependencies, and understand blast radius from the state layer itself.

Infrastructure configuration management stops being just a file you apply and becomes operational data you can reason about.

Auditability follows from the same model. Stategraph keeps a transaction timeline that records Terraform operations and state changes, including who changed what and when, then connects that history to drift detection and PR automation so infrastructure state can participate in the same compliance and observability workflows as the rest of the software delivery system.

State should not be a blob that teams lock, rewrite, and hope they can explain later. It should tell you what changed, who changed it, when it changed, and which resources were affected.

This is the next step for teams using Terraform or OpenTofu that have already done the obvious things. They have version control. They have CI. They have CD. They have automated testing. They have IaC. They have monitoring. The remaining problem is that the state layer has not caught up with the maturity of the rest of the workflow.

Conclusion

DevOps best practices are well understood now. Most serious software teams know they need version control, continuous integration, continuous delivery, automated testing, infrastructure as code, monitoring, incident processes, and continuous feedback.

The gap is no longer between teams that have heard of DevOps and teams that have not. The gap is between teams that have adopted DevOps practices and teams whose practices keep working as the organization, infrastructure, and delivery cadence grow.

For infrastructure-focused teams, that often comes down to state. If Terraform or OpenTofu state is serialized behind global locks, if drift detection competes with deployment pipelines, if infrastructure inventory is not queryable, and if audits depend on reconstructing history from scattered tools, then the DevOps workflow is only mature above the waterline.

Stategraph is built for the layer underneath. For teams that want state management to match the maturity of the rest of their DevOps workflow, try Stategraph free.

DevOps best practice FAQs

What are the 7 DevOps practices?

The seven most commonly cited DevOps practices are version control, continuous integration, continuous delivery, continuous deployment, automated testing, infrastructure as code, and monitoring and observability. Different teams group them differently, and some add security, incident response, or continuous feedback as separate practices, but those seven cover the core DevOps lifecycle most teams mean when they talk about DevOps best practices.

What two practices is DevOps bringing together?

DevOps brings together software development and IT operations. The point is shared ownership of the software development lifecycle, from code changes and testing through deployment, monitoring, incident response, and continuous improvement.

How are DevOps practices best orchestrated?

DevOps practices are best orchestrated as a connected system, with shared ownership across development and operations teams, automation as the default path through the pipeline, and observability that spans application behavior, deployment history, infrastructure state, and change history. Tooling helps, but orchestration comes from the workflow. Tools only make that workflow executable.