Layered Runs

Layered runs plan and apply dependent directories in order. Stategraph Orchestration runs each layer only after the layers that it depends on apply successfully. For example, a network must exist before the database in it, and the database must exist before the application that connects to it.

How it works

The depends_on key in the when_modified configuration of a directory names the directories that must apply first. See the when_modified reference.

The DIR variable

${DIR} in file_patterns expands to the directory that the dirs entry matches. For the key envs/asia/database, ${DIR} is envs/asia/database. See the dirs reference.

Add this to .stategraph/config.yml:

dirs:
  network:
    when_modified:
      file_patterns: ["${DIR}/*.tf"]
  database:
    when_modified:
      depends_on: 'dir:network'
      file_patterns: ["${DIR}/*.tf"]
  application:
    when_modified:
      depends_on: 'dir:database'
      file_patterns: ["${DIR}/*.tf"]

With this configuration:

  1. If network changes, Orchestration plans and applies it first.
  2. After the network apply succeeds, Orchestration plans and applies database.
  3. After the database apply succeeds, Orchestration plans and applies application.

If a layer has no changes, its plan comment says so and tells you when the next layer follows.

Dependency expressions

depends_on is a tag query, so it accepts logical operators and two directory prefixes.

Logical operators

Combine dependencies with or and and. A change in any directory that the query names triggers the dependent layer.

dirs:
  application:
    when_modified:
      depends_on: 'dir:network or dir:database'

Here a change to network or to database triggers application. Set depends_on under dirs.<directory>.when_modified. Under the top-level when_modified, it applies to every directory, the dependencies included.

Directory references

dir: names an absolute path from the repository root. Use it when the directory layout is fixed.

dirs:
  database:
    when_modified:
      depends_on: 'dir:network'

relative_dir: names a path relative to the current directory. Use it when the same rule applies to many environments, or when the layout can move.

dirs:
  envs/prod/database:
    when_modified:
      depends_on: 'relative_dir:../network'

Pruning layers without changes

By default, when a dependency changes, the dependent layer joins the run even if it has no changes of its own. To keep a layer in the order without running it on each upstream change, use the object form of depends_on and set prune_on_no_change: true. The layer then runs only when it has changes of its own, and it still orders the layers that depend on it.

dirs:
  database:
    when_modified:
      depends_on:
        tag_query: 'dir:network'
        prune_on_no_change: true
      file_patterns: ["${DIR}/*.tf"]

Here database is ordered after network, but a change to network alone does not plan or apply database.

Layers and stacks

With a stacks section, a depends_on can name only directories in the same stack, or in a stack that shares a nested stack with it. A directory in an unrelated stack is a configuration error. To order two unrelated stacks, use the stack rules plan_after and apply_after.

Use cases

Sequential infrastructure deployment

Layers that depend on each other must deploy in order: when you rebuild from scratch, for example in disaster recovery, and when you change one layer.

dirs:
  network:
    when_modified:
      file_patterns: ["${DIR}/*.tf"]
  database:
    when_modified:
      depends_on: 'dir:network'
      file_patterns: ["${DIR}/*.tf"]
  application:
    when_modified:
      depends_on: 'dir:database'
      file_patterns: ["${DIR}/*.tf"]

Multiple layers with shared dependencies

app1 and app2 both depend on a shared database, which depends on network. A change to network reaches database, then both applications. shared_resources depends on either of the two lower layers.

dirs:
  network:
    when_modified:
      file_patterns: ["${DIR}/*.tf"]
  database:
    when_modified:
      depends_on: 'dir:network'
      file_patterns: ["${DIR}/*.tf"]
  app1:
    when_modified:
      depends_on: 'dir:database'
      file_patterns: ["${DIR}/*.tf"]
  app2:
    when_modified:
      depends_on: 'dir:database'
      file_patterns: ["${DIR}/*.tf"]
  shared_resources:
    when_modified:
      depends_on: 'dir:network or dir:database'
      file_patterns: ["${DIR}/*.tf"]

One configuration for every environment

When environments share a layout, define each layer once with a glob and relative_dir:. In this repository, four environments each hold some of the services application, database, networking, and block_storage.

.
└── envs
    ├── asia
    │   ├── application
    │   │   └── main.tf
    │   ├── database
    │   │   └── main.tf
    │   └── networking
    │       └── main.tf
    ├── europe
    │   ├── application
    │   │   └── main.tf
    │   ├── block_storage
    │   │   └── main.tf
    │   ├── database
    │   │   └── main.tf
    │   └── networking
    │       └── main.tf
    ├── us-east
    │   ├── application
    │   │   └── main.tf
    │   ├── database
    │   │   └── main.tf
    │   └── networking
    │       └── main.tf
    └── us-west
        ├── application
        │   └── main.tf
        ├── block_storage
        │   └── main.tf
        ├── database
        │   └── main.tf
        └── networking
            └── main.tf

The layers, from bottom to top:

  1. networking
  2. database and block_storage
  3. application
dirs:
  'envs/*/database':
    when_modified:
      depends_on: 'relative_dir:../networking'
  'envs/*/block_storage':
    when_modified:
      depends_on: 'relative_dir:../networking'
  'envs/*/application':
    when_modified:
      depends_on: 'relative_dir:../database or relative_dir:../block_storage'

If envs/asia/database changes, envs/asia/application runs after it. If envs/us-west/networking changes, envs/us-west/database and envs/us-west/block_storage run next, then envs/us-west/application.

Only two environments have block_storage. The rule applies only to the directories that exist.

Best practices

  • Make depends_on match real dependencies. Orchestration cannot order circular dependencies.
  • Combine depends_on with tag queries to trigger only the layers that a change needs.
  • Split infrastructure into layers that you can manage separately. Small layers keep the order simple and the runs short.

Next steps