Security best practices

These practices limit what a Stategraph Orchestration run can reach and who can start it.

Mitigating risks from untrusted HCL

Orchestration runs terraform plan on your CI runners for each pull request, and a plan of untrusted HCL is code execution. With the external provider, null_resource provisioners, or other providers, anyone who can open a pull request can run commands on the runner. That gives them the credentials that the runner holds for the plan, and through them, access to your production environment.

Isolate secrets

Combine GitHub Environments with access control, so that only runs from trusted people reach production secrets:

workflows:
  - tag_query: 'dir:production'
    environment: production
access_control:
  enabled: true
  apply_require_all_dirspace_access: true
  plan_require_all_dirspace_access: false
  terrateam_config_update: ['team:admins']
  unlock: ['team:admins']
  policies:
    - tag_query: 'dir:production'
      plan: ['team:developers']
      apply: ['team:sre']

This configuration:

  • binds the production workflow to the production GitHub Environment, so its secrets and variables are available only to that workflow
  • requires access to each targeted dirspace for an apply
  • lets only the admins team change the configuration and unlock
  • gives developers plan access and sre apply access for production

Access control is an Enterprise feature. See Editions.

Enforce policy from one place

Use centralized configuration to set defaults for each repository in the organization, and policy that the .stategraph/config.yml of a repository cannot override.

Secure workflow design

Use OIDC

Avoid long-lived static keys. The oidc step issues credentials that expire after the run:

workflows:
  - tag_query: 'dir:production'
    plan:
      - type: oidc
        provider: aws
        role_arn: ${AWS_ROLE_PROD_ARN}
      - type: init
      - type: plan
    apply:
      - type: oidc
        provider: aws
        role_arn: ${AWS_ROLE_PROD_ARN}
      - type: init
      - type: apply

See Cloud credentials and Hardening AWS OIDC.

Enforce policy before apply

Check each plan against policy with Open Policy Agent. This workflow copies policies from a private bucket and runs conftest against the plan:

workflows:
  - tag_query: 'dir:production'
    plan:
      - type: oidc
        provider: aws
        role_arn: ${AWS_ROLE_PROD_ARN}
      - type: run
        cmd: ['aws', 's3', 'sync', 's3://acme-sre-private-bucket/opa/policies/', '/tmp/policies/']
      - type: init
      - type: plan
      - type: env
        name: CONFTEST_POLICY
        cmd: ['echo', '/tmp/policies/']
      - type: conftest

A failed conftest step fails the plan, so you cannot apply a change that does not comply. See OPA and Security scanning.

Detect drift

Turn on drift detection to find changes made outside Terraform:

hooks:
  plan:
    post:
      - type: drift_create_issue
drift:
  enabled: true
  schedules:
    default:
      tag_query: ''
      schedule: daily

This runs drift detection daily, and opens an issue in the repository when it finds drift. On GitLab, the issue needs an access token in the TERRATEAM_DRIFT_ACCESS_TOKEN CI/CD variable.

Require reviews before apply

Set apply requirements for production:

apply_requirements:
  checks:
    - tag_query: 'dir:production'
      approved:
        enabled: true
        any_of_count: 2
      merge_conflicts:
        enabled: true
      status_checks:
        enabled: true

A production apply then needs two approvals, no merge conflicts, and all status checks passing.

Operational practices

  • Run each infrastructure change through the pull request flow, so that plans, approvals, and applies are recorded.
  • Watch runs for unexpected activity. In the console, Runs under Orchestration lists each run. The audit trail queries the history by user, directory, and date. The job logs in GitHub Actions or GitLab CI hold the runner detail.
  • Alert on failed runs and drift with notifications and the audit trail.
  • Store secrets as GitHub Actions secrets, GitLab CI/CD variables, or in a secrets manager. Rotate keys and tokens on a schedule.
  • Store plan files where access is restricted. See Plan file storage.
  • Regularly review who can plan, apply, unlock, and change the configuration, and give each role the least privilege that it needs. Review your configuration and workflows when the organization changes.

In this section

Governance:

Console access:

  • Access control: authentication, tenants, access tokens, and group rules for the console and API.