Locks and Concurrency

Stategraph Orchestration locks the directories and workspaces of a pull request, so that only one pull request at a time applies a change to them. Locks keep the real infrastructure consistent with the code on your default branch.

Why locks matter

Suppose Jane and John both open pull requests that change the same Terraform resource. Without locks, both can apply, and the infrastructure no longer matches the code on main. With locks:

  1. Jane's pull request applies and takes the lock.
  2. John's apply is blocked while Jane holds the lock.
  3. Jane's pull request merges, and the lock is released.
  4. John plans again against the new main, applies, takes the lock, and merges.

How locks work

A lock belongs to a pull request and covers a dirspace: a directory and a workspace. While a pull request holds a lock, no other pull request can apply to that dirspace. By default:

  • A pull request takes a lock when it applies, or when it merges without an apply. The lock covers the dirspaces that the pull request targets.
  • A pull request takes the lock whether the apply succeeds or fails.
  • After one dirspace of a pull request applies, the pull request holds locks on every dirspace that it targets.
  • A pull request that has not applied or merged holds no lock.
  • A lock is released only after a successful apply and a merge.

Example

A pull request changes dir1 and dir2, both in the default workspace:

  1. The pull request opens.
  2. stategraph plan runs.
  3. Someone comments stategraph apply dir:dir1.
  4. The pull request now holds locks on (dir1, default) and (dir2, default).

Unlocking

To remove a lock by force, comment stategraph unlock on the pull request that holds it. The unlock rule of access control sets who can unlock.

Unlock scope

stategraph unlock removes only the locks taken before the comment. A merge or an apply after the unlock takes locks again.

Use with care

stategraph unlock removes every lock that the pull request holds. When several pull requests comment stategraph apply, Orchestration runs only one apply at a time. If you unlock while an apply runs, the locks are released, and a second pull request can apply against the same resources. If both pull requests change the same resources, the second apply can have unintended results.

Lock policies

The lock_policy key sets when Orchestration takes a lock. Set it at the top level for the whole repository, and on a workflow entry to override it for the directories that the workflow matches.

Policy Description Recommended use
strict Takes a lock on a stategraph apply comment or on merge. Releases it when the other action happens. Production environments (default)
apply Takes a lock only when the directory applies. Releases it when the change merges. Standard apply-then-merge workflows
merge Takes a lock only when the directory merges. Releases it when the change applies. Development and playground branches
none Never takes a lock. Special cases only. Not recommended.

To use apply for the whole repository and keep strict for production:

lock_policy: apply

workflows:
  - tag_query: dir:production
    lock_policy: strict

Any value other than strict is less safe. Use it only where a stale infrastructure state is acceptable.

Locks in Infrastructure as a Database

Orchestration locks are per pull request and per dirspace. Infrastructure as a Database also locks single resources in a state, so two transactions that change different resources in the same state do not conflict. See Resource-level locking.

Best practices

  • Tell your team when you work on shared directories, so that applies do not wait for each other.
  • Check long-held locks regularly. An apply that never merges blocks everyone else on that directory.
  • Use stategraph unlock rarely, and make sure that the whole team knows what it does.
  • Agree on a process to take, release, and force locks before the first conflict.

Next steps