Terraform State: A Practical Guide to Backends, Locks and Safe CI/CD

The difference between teams that ship confidently and teams that fear every apply comes down to how they handle state.

This guide covers everything you need to know about Terraform state for production environments: what your Terraform state file actually contains, how to configure remote backends properly, how to inspect and edit state safely with the CLI, why locking matters (and how to unlock it when it gets stuck), how to secure sensitive data, and how to integrate state management into CI/CD without creating bottlenecks or security holes.

What is Terraform state?

Terraform state is a JSON file that maps your configuration code to real infrastructure resources. When you run terraform apply for the first time, Terraform creates a terraform.tfstate file: your Terraform state file in your working directory. This file becomes Terraform's database of what exists and where.

Without state, Terraform cannot determine what infrastructure already exists or what needs to change. The state file records resource IDs, attributes, dependencies, and outputs. It's the binding between your declarative configuration and the actual resources running in your cloud provider.

Every state file contains several critical components:

• Resource mappings connect your aws_instance.web_server to EC2 instance i-01234abcd, providing the one-to-one mapping that lets Terraform know exactly which real-world resource corresponds to which line of code.
• Dependency metadata ensures operations happen in the correct order (Terraform won't delete a security group that an EC2 instance depends on because the state file tracks these relationships).
• Outputs allow other configurations or automation tools to query values from your infrastructure, stored in state and available for remote reference.
• Serial and lineage provide state versioning, where the serial number increments with each change and the lineage ID uniquely identifies the state file's history, preventing conflicting updates from mixing incompatible state histories.

Open a state file directly (something you'll rarely need to do, but it helps to have actually seen one), and the structure matches what's described above.

A minimal example for a single EC2 instance looks like this:

The resources array is what has the greatest impact day to day. Each entry records the resource's type, name, and provider, then an instances array holding the attributes Terraform last recorded: the actual AMI, instance type, and ID that exist in AWS right now (or did, at the last apply).

serial increments on every write, and lineage is the state's unique identity; mismatched lineage between a local copy and the remote backend is usually the first clue that someone is looking at the wrong state entirely.

State lifecycle

Before every plan or apply, Terraform refreshes state by checking actual infrastructure for changes. If someone manually terminated a VM outside Terraform, the refresh detects it and updates state accordingly.

After a successful apply, Terraform writes a new state snapshot and saves the previous version as terraform.tfstate.backup.

This lifecycle is automatic. You don't manually edit state files. Instead, you use Terraform CLI commands that handle state modifications safely and maintain format compatibility across versions.

Why local state fails at scale

The default local state file works for solo projects and learning, but it fails when you add teammates or automation. Local state creates several problems that remote backends solve.

Local state has:

• No collaboration. When state lives on your laptop, nobody else can run Terraform, and if you're on vacation and production needs an emergency change, your team is stuck.
• No locking. If two people somehow share a state file via Dropbox, Git, or a network drive, concurrent runs will corrupt state as there's no coordination mechanism to prevent simultaneous writes.
• No durability. Laptop crashes, accidental deletions, and disk failures cause permanent state loss. Without state, Terraform thinks nothing exists and will try to recreate everything.
• No history. Local state keeps one backup file, so if you need to roll back further or audit changes, you're out of luck.

Remote backends store state in a shared, durable location. All major cloud platforms offer backends that Terraform can use: S3 on AWS, Blob Storage on Azure, and Cloud Storage on GCP.

These backends add locking, versioning, encryption, and access control that local state cannot provide.

Configuring remote backends

Using a remote backend requires two steps: configure the backend block in your Terraform code (written in HashiCorp Configuration Language, or HCL) and run terraform init to migrate state. Below you'll find configurations for each major cloud provider.

S3 backend (AWS)

S3 provides durable object storage with versioning and encryption. A production S3 backend configuration looks like this:

Enable S3 bucket versioning to recover from accidental deletions or corrupted state. Versioning keeps every state update as a separate object version, giving you a complete history.

The encrypt = true flag enables server-side encryption. Your state file contains resource IDs, IP addresses, and sometimes secrets. Encryption at rest is not optional.

For state locking, Terraform supports native S3 locking via use_lockfile = true, which creates a .tflock object in the bucket to coordinate concurrent access.

Older configurations relied on a separate DynamoDB table for locking, an approach that is now deprecated in Terraform's own S3 backend documentation and scheduled for removal in a future minor version, so new deployments should use use_lockfile rather than standing up a DynamoDB table.

Credentials and access: Never hardcode AWS keys in your backend config. Use environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) or IAM roles. For CI/CD, configure the pipeline to assume an IAM role with minimal permissions: s3:GetObject, s3:PutObject, and s3:ListBucket on the specific state bucket and path only.

Azure Blob Storage backend

Azure Storage accounts provide blob containers for state storage. Native locking via blob leases handles concurrency automatically.

Azure Storage encrypts data at rest by default. Restrict access via Azure RBAC or SAS tokens so only authorized users and service principals can read or write state. Disable public access on the storage account entirely and consider private endpoints to limit network exposure.

The AzureRM backend handles locking automatically using blob leases. When Terraform writes state, it acquires a lease on the blob, preventing other processes from writing simultaneously. You won't require any additional coordination service.

Authentication: Use managed identities or service principals instead of hardcoding credentials. Supply authentication via environment variables or Azure CLI login instead of embedding secrets in your Terraform configuration.

Google Cloud Storage backend

GCS buckets store state with automatic locking via generation numbers and preconditions. Enable object versioning to keep historical state versions.

Terraform places the state file under the specified prefix path. The workspace name gets appended automatically, so the default workspace creates terraform/state/prod/default.tfstate.

GCS encrypts data at rest by default. For additional security, supply a customer-managed encryption key if one is required by your organization's policies.

Access control: Use Cloud IAM to restrict the bucket. Grant the service account or user running Terraform roles/storage.objectAdmin on the specific bucket or prefix. Ensure no public access. Handle credentials via GOOGLE_APPLICATION_CREDENTIALS environment variable or gcloud application-default credentials.

Backend migration

Moving from local to remote state is straightforward. Add the backend block to your configuration and run:

Terraform detects the backend change, prompts for confirmation, and copies your local state to the remote backend. After migration, delete your local state file and rely entirely on the remote backend as the source of truth.

Inspecting and editing state with the CLI

Terraform's state subcommands let you look inside a state file and make surgical corrections without hand-editing JSON, which the lifecycle rules above should have already convinced you never to do.

Four commands cover almost everything you'll need:

• terraform state list to see what's there
• terraform state show to inspect one resource closely
• terraform state mv to reorganize without losing history
• terraform state rm to cut a resource loose from Terraform's memory.

Newer HCL blocks (moved, import, and removed) do the same jobs declaratively, and are worth using initially when your Terraform or OpenTofu version supports them.

terraform state list

terraform state list prints every resource address Terraform currently tracks, the fastest way to confirm a resource exists in state before you try to inspect or move it. For a state containing a VPC with three subnets, the output looks like this:

Filter by a module or resource prefix on larger states (terraform state list module.network) rather than scrolling through hundreds of addresses looking for one resource.

terraform state show

terraform state show prints everything Terraform recorded about a single resource. Pass it the resource address (the same address state list gives you), and it reads straight from state, no provider API call involved.

That code is the entire command.

It's fast because it isn't asking AWS anything; it's reading Terraform's own record. That speed comes with a ceiling: terraform state show tells you what a resource looks like, not what depends on it or whether a change to it will break something three resources downstream.

If you're debugging a specific attribute value after an apply, or confirming an import landed at the right address with the right ID, it's the right tool. If you're trying to work out the impact of touching a shared resource before you touch it, it's the wrong one.

terraform state mv

terraform state mv moves a resource from one address to another inside state without touching the real infrastructure behind it. Use it when you rename a resource, pull a flat resource into a module, or need the address in state to match a refactored configuration.

Terraform keeps the resource's full history at the new address; nothing gets destroyed and recreated. The command also moves resources between separate state files entirely (terraform state mv -state-out=../other.tfstate ...), which is the mechanism behind most state-splitting and ownership-handoff workflows, covered further down.

terraform state rm

terraform state rm removes a resource's record from state without calling the provider to delete anything. The resource keeps running in AWS, Azure, or wherever it lives; Terraform simply forgets it manages that address.

That distinction, state forgets versus infrastructure disappears, is the entire point of the command, and it's also where teams get into trouble.

Removing a resource from state while its resource block stays in your configuration doesn't delete anything; it just means the next plan proposes creating a second copy of something that already exists, because Terraform no longer has a record to compare against.

Use terraform state rm in migration and handoff scenarios: pulling a resource out of one team's state so another team can run the terraform import command against the same object in theirs, or cleaning up a stale entry after a resource was deleted manually outside Terraform. It's a poor substitute for terraform destroy when the goal is actually deleting infrastructure, and also a poor substitute for terraform state mv when the goal is simply relocating a resource inside the same overall estate.

Declarative alternatives: moved, import, and removed blocks

Modern Terraform ships moved, import, and removed blocks, which do in HashiCorp Configuration Language what the CLI commands above do imperatively, with one real advantage: you can preview the change with terraform plan before anything happens to state.

Use the removed block instead of the terraform state rm command as it lets you preview the result first.

A moved block does what terraform state mv does, but shows up in your plan output and lives in version control, so a teammate reviewing the pull request sees the refactor coming rather than discovering it after the fact. removed blocks work the same way for the state rm case:

Setting destroy = false tells Terraform to drop the resource from state without touching the real infrastructure, the declarative equivalent of terraform state rm. Use these blocks if your Terraform or OpenTofu version supports them, and save the raw CLI commands for the cases HCL blocks don't cover, or for a configuration pinned to an older version.

terraform refresh and drift

terraform plan and terraform apply both refresh state against real infrastructure automatically before doing anything else, but sometimes you want that refresh on its own, without a plan or apply attached.

terraform refresh (or terraform apply -refresh-only on newer versions, which at least shows you a plan-shaped diff before writing anything) updates state to match reality without changing your configuration or your infrastructure.

Refresh looks for and finds drift: a resource attribute that no longer matches what state recorded, usually because someone changed something through a cloud console instead of through Terraform.

Once you've confirmed the drift, you have two ways to resolve it. Update your configuration to match what changed, if the manual change was legitimate and should stick, or run terraform apply and let Terraform revert the drift back to what your configuration says, if the manual change was a mistake that should be superseded by the real workflow.

Why state locking is important

State locking prevents concurrent modifications that corrupt state. When multiple engineers or CI jobs run Terraform simultaneously, if you don't have locking, you get race conditions where writes overwrite each other, leaving state inconsistent or broken.

Terraform's locking mechanism is simple. For backends that support it, Terraform automatically acquires a lock before any write operation. If a lock already exists, Terraform waits until it's released. Only one process can hold the lock at a time.

All major remote backends support locking: S3 with native lock files, Azure with blob leases, GCS with object locking, HCP Terraform (formerly Terraform Cloud) with automatic locking. Local backends do not support locking, which is one reason they fail in team environments.

How locking works

When you run terraform apply, Terraform attempts to acquire a lock before making changes. This happens automatically. You don't see it unless there's contention.

If another process holds the lock, Terraform waits and displays a message like "Waiting for state lock." Once the lock releases, your operation proceeds. After finishing, Terraform releases the lock automatically.

Terraform provides a -lock=false flag to bypass locking, but using it is risky. Only disable locking in emergencies when you're certain no other process is running. The correct approach to lock contention is to fix the coordination problem, not disable the safety mechanism.

How to unlock Terraform state

You can unlock a Terraform state file with a one-command fix once you've done the diagnostic work first; skipping that step is how a stuck lock turns into corrupted state. If Terraform crashes or a process terminates abnormally, the lock might not release, and your next run will fail with a lock error that displays a lock ID.

First, verify no Terraform process is actually running. Check your CI jobs, ask your teammates, and ensure nothing's applying. Then force-unlock using the lock ID from the error message:

Use force-unlock carefully. The lock ID acts as a safety check to prevent anyone accidentally unlocking a different lock. If you see frequent stuck locks, fix the root cause (e.g., crashed processes, timeout issues, or interrupted CI jobs) rather than routinely relying on for force-unlock.

Security best practices

State files contain sensitive information: resource IDs, IP addresses, database connection strings, and sometimes secrets in plaintext. Securing state is not optional for production environments.

Encryption at rest

Always encrypt state files at rest. Enable server-side encryption on your backend storage.

• For S3, use encrypt = true in your backend config.
• For Azure and GCS, encryption is enabled by default, but verify it's active and consider customer-managed keys for additional control.

Encryption in transit happens automatically via TLS when Terraform communicates with the backend. Ensure you're using HTTPS endpoints, never unencrypted HTTP.

Access control

Restrict who can read or write state. Use IAM policies, bucket policies, or RBAC to limit access to the state storage location. Only Terraform processes and administrators should have access.

• For AWS S3, grant minimal permissions to the CI/CD role: s3:GetObject, s3:PutObject, s3:ListBucket on the specific bucket and path. Block public access entirely.
• For Azure, use RBAC to grant the appropriate AAD principals access. Disable public access on the storage account and consider private endpoints to limit network exposure.
• For GCS, grant roles/storage.objectAdmin on the specific bucket or prefix only. Ensure no public access.

Credential handling

Never hardcode credentials in backend configurations. Use environment variables, IAM roles, managed identities, or service principals. Embedding secrets in your Terraform code could mean they end up in version control and CI logs.

• For AWS, use IAM roles so no static keys are required.
• For Azure, use managed identities or service principals with credentials supplied via environment.
• For GCP, use application-default credentials or service account key files referenced via environment variables.

Terraform's backend configuration supports partial configuration, allowing you to omit sensitive fields from the config and supply them via environment or command-line flags.

Never commit state to Git

This is a common anti-pattern. State files contain secrets and should never go in version control. If you accidentally commit state, you must purge it from Git history and rotate any exposed credentials.

Add *.tfstate and *.tfstate.backup to your .gitignore immediately. Use remote backend versioning for state history, not Git.

Sensitive data in state

Terraform stores all resource attributes in state, including sensitive values. Marking outputs as sensitive = true prevents them from displaying in CLI output, but they're still stored in plaintext in the state file. You can see why state file encryption and access control are so important here

Some teams avoid putting highly sensitive secrets in Terraform-managed resources entirely, instead using HashiCorp Vault or cloud secret managers for dynamic secret injection.

Weigh the convenience of Terraform-managed secrets against the exposure risk. For production systems with strict compliance requirements, consider external secret management integrated with Terraform rather than embedding secrets in configurations.

State management in CI/CD

To successfully integrate Terraform into CI/CD pipelines, you need careful state management. Pipelines run in ephemeral environments, so remote state is mandatory. Concurrent pipeline runs need locking. Credentials need secure injection. Get any of this wrong, and you can create security holes or broken state.

Always use remote state

When Terraform runs in CI, the pipeline environment doesn't retain local files between runs. Without a remote backend, each run starts from scratch and treats existing infrastructure as new, trying to recreate everything.

Configure your CI jobs to use the same remote backend as developers. The pipeline initializes with terraform init, pulls the latest state from the backend, runs plan or apply, and pushes state updates back.

Provide secure credentials

CI jobs need credentials to access the remote state backend. Use your CI platform's secret management to inject credentials as environment variables at runtime.

• For AWS, configure the CI job to assume an IAM role with permissions to the S3 bucket (and a DynamoDB table, if you're still on the older locking pattern).
• For Azure, use a service principal with RBAC permissions to the storage account.
• For GCP, use a service account key stored in CI secrets and injected via GOOGLE_APPLICATION_CREDENTIALS.

Never put credentials in your Terraform configuration or CI pipeline definition files. They should come from secure secret stores and only exist as environment variables during pipeline execution.

Handle locking and concurrency

In busy environments, multiple pipeline runs can trigger simultaneously. State locking serializes the applies to prevent conflicts. However, you should also configure your CI orchestrator to handle concurrency intelligently.

Some CI systems allow queueing jobs per environment or setting concurrency limits. Use these features to prevent multiple applies from constantly fighting for the lock. Terraform's lock will work, but a better approach is pipeline-level coordination so only one job runs at a time per state.

If you are using HCP Terraform's remote runs, it handles queueing automatically. For self-managed CI, configure job concurrency limits per environment to reduce lock contention.

Separate state per environment

Your CI pipelines likely deploy to multiple environments: dev, staging, and production. Each environment must use separate state to prevent accidental cross-environment changes.

Consider separate backend configurations per environment, different state file keys or prefixes, or Terraform workspaces. For example, your production pipeline uses key = "prod/terraform.tfstate" while staging uses key = "staging/terraform.tfstate".

This isolation ensures a deployment to dev doesn't accidentally read or write prod's state, reducing blast radius and enabling parallel development across environments.

Terraform workspaces as an alternative

Terraform workspaces are a second way to isolate state, alongside or instead of separate backend keys. Each workspace gets its own state file under the same backend configuration, switched with terraform workspace select.

Workspaces are quick to set up and convenient for short-lived, structurally identical environments (spinning up an ephemeral environment for a feature branch, say).

However, you should be cautious about using workspaces as your only mechanism for separating permanent environments like staging and production. Workspaces share the same backend configuration and the same variable definitions by default, which makes it easy for a terraform.workspace conditional buried in a module to become the only thing standing between a plan and a production change.

Separating backend configurations per environment, which is covered above, gives you a harder boundary: different credentials and access policies per environment, with no shared configuration path between environments at all.

Sharing state between configurations

Splitting infrastructure across multiple state files (by environment, by team, or by the boundaries covered next) creates a new problem: one configuration sometimes needs a value that lives in another configuration's outputs, a VPC ID or a database endpoint, say.

The terraform_remote_state data source reads another state file's outputs directly:

It works, but it grants read access to the entire source state file, not just the specific output you need, which means every secret and every resource attribute in the network state is now reachable from the compute configuration too.

Where you can, opt for a narrower data source instead (an aws_vpc or aws_subnet data source, for instance) that reads the live resource directly rather than the whole state file behind it, and reserve terraform_remote_state for outputs with no equivalent direct data source.

Plan and apply stages

Many pipelines split Terraform into separate plan and apply stages with manual approval in between. Both stages must use the same remote state.

The plan stage runs terraform plan -out=tfplan and saves the plan file as a pipeline artifact. The apply stage runs terraform apply tfplan using the exact plan from the previous stage.

Between plan and apply, state could change if someone else runs Terraform. The apply will detect this and fail, prompting a re-plan. Some teams implement additional checks or short-lived locks, but Terraform's built-in refresh on apply provides a safety baseline.

Avoid storing state in pipeline artifacts

Rely on the remote backend as the source of truth, not pipeline artifacts. Saving state files between pipeline jobs creates confusion and risks applying with stale state.

If you need to pass information to subsequent jobs, use terraform output -json to extract outputs after apply rather than passing the raw state file around.

Common pitfalls and how to fix them

Even with best practices, teams encounter state issues. Here are the most common problems and their solutions.

State file corrupted or lost

If your state file gets corrupted or accidentally deleted, and you have versioning enabled on your backend, retrieve the last good version.

• For S3, use the version history in the AWS console or CLI to restore a previous state version.
• For Azure and GCS, similar version recovery is available.
• For HCP Terraform, state history is built-in.

If you have no backups, you'll need to reconstruct state by importing resources. Use the terraform import command to bring existing resources under Terraform management by mapping them to your configuration.

This is tedious for large infrastructures, which is why backend versioning is critical. Always enable it.

Drift between state and reality

Resources sometimes change outside Terraform when someone modifies infrastructure manually via the cloud console. Terraform detects this during the refresh phase of plan or apply.

Run terraform plan to see what differs between state and reality. Terraform will show changes needed to bring reality back in line with your configuration.

If you want reality to come out on top (adopting the manual change), update your configuration to match what currently exists, then run apply. If you want your configuration to win (reverting the manual change), just apply and Terraform will fix the drift.

Stuck lock won't release

If you get a lock error, first confirm no other Terraform process is running. Then, use terraform force-unlock <LOCK_ID> with the ID from the error message.

If this happens frequently, investigate why processes are crashing or getting interrupted. Fix the root cause rather than routinely force-unlocking.

Resource already exists

This occurs when you try to create a resource that already exists, often because it was provisioned outside Terraform or is managed in a different state file.

The fix is importing the existing resource rather than trying to create it anew. Use terraform import to bring it under Terraform management in your current state.

If the resource exists in two different state files (which is a coordination issue), remove it from one using terraform state rm to maintain the one-to-one mapping principle. Each real resource should be managed by exactly one Terraform state.

State file too large

A state file that holds thousands of resources makes terraform plan and terraform apply slower, because Terraform refreshes and diffs the entire file on every run, and it raises the odds of two engineers colliding over the same lock.

Splitting state into smaller files by environment or application (or another functional boundary that fits the estate), moved with terraform state mv or set up as a fresh backend per component, is the standard fix teams use, and it does shrink the blast radius of any one change.

However, it comes with its own issues, not least adding coordination overhead. Configurations now reach across terraform_remote_state lookups to talk to each other, and drift has more places to hide. Lock contention doesn't disappear so much as relocate: from the whole state to whichever smaller state everyone happens to be touching that day.

Stategraph takes a different approach to these two underlying causes (slow plans on large states, and multiple engineers blocked on the same lock): rather than splitting infrastructure into separate files to shrink what each apply touches, graph-aware execution operates only on the subgraph actually affected by a change and locks at the resource level instead of the file level, across your existing states.

Splitting state is an option when your infrastructure should have genuinely separate ownership and blast radius in principle. However, it's unnecessary for most teams that need just to make plans faster or unblock two engineers working on unrelated resources.

Manual state editing

Never manually edit state files. A JSON formatting error can corrupt the entire state, and removing a resource from state doesn't destroy the real resource.

Instead, use Terraform's state subcommands: terraform state list to show all resources in state, terraform state show <resource> to display a resource's attributes, terraform state rm <resource> to remove a resource from state without destroying it, and terraform state mv <source> <dest> to rename or move a resource within state.

These commands operate safely on state without altering real infrastructure. Use them for cleanups, renames, and migrations. When in doubt, back up state first (most backends provide versioning for this).

The bottom line

Terraform state management is the foundation of infrastructure as code (IaC) that determines whether your workflows are reliable or fragile.

• Use remote backends with encryption, versioning, and access controls.
• Enable state locking to prevent concurrent modifications.
• Never commit state to Git.
• Handle credentials securely via IAM roles, managed identities, or environment variables.
• Integrate state management properly into CI/CD with remote backends, secure credential injection, and concurrency controls.
• Know how to recover from common issues, and how to inspect, move, or unlock state, using Terraform's own state subcommands rather than editing the file by hand.

State is Terraform's database of what exists. Treat it accordingly. The teams that get this right ship confidently. The teams that ignore it spend their time firefighting corrupted state and explaining outages.