Gap Analysis

Gap analysis discovers resources in your cloud provider that exist but aren't managed by any Terraform state in Stategraph.

Overview

As infrastructure grows, resources can be created outside of Terraform:

• Manual console changes
• Scripts or CLI commands
• Other IaC tools
• Forgotten experiments

Gap analysis compares your cloud inventory against Terraform state to find these unmanaged resources.

How It Works

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  Cloud Provider │    │   Stategraph    │    │   Gap Report    │
│    Inventory    │───▶│    Compare      │───▶│   Unmanaged     │
│                 │    │                 │    │   Resources     │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                                            │
         │                                            │
  AWS Resource Explorer                         • S3 bucket
  GCP Cloud Asset Inventory                     • IAM role
                                               • Compute instance

1. Fetch inventory - Query cloud provider for all resources
2. Compare with state - Match against resources in Terraform states
3. Report gaps - List resources not found in any state

Supported Providers

AWS

Gap analysis uses AWS Resource Explorer to discover resources.

Setup Requirements

1. Resource Explorer must be enabled with at least one index
2. Aggregator index recommended for cross-region discovery
3. Default view must be configured

Required IAM Permissions

For gap analysis (read inventory):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "resource-explorer-2:Search",
        "resource-explorer-2:ListIndexes",
        "resource-explorer-2:GetDefaultView"
      ],
      "Resource": "*"
    }
  ]
}

For generating Terraform configurations (optional):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "arn:aws:iam::aws:policy/ReadOnlyAccess"
      ],
      "Resource": "*"
    }
  ]
}

Enable Resource Explorer

# Enable Resource Explorer in current region
aws resource-explorer-2 create-index --type LOCAL

# Create aggregator index (recommended for multi-region)
aws resource-explorer-2 update-index-type \
  --arn "arn:aws:resource-explorer-2:us-east-1:123456789012:index/..." \
  --type AGGREGATOR

# Create default view
aws resource-explorer-2 create-view --view-name default
aws resource-explorer-2 associate-default-view --view-arn "arn:aws:resource-explorer-2:..."

Check Configuration

# Set API base (or use --api-base flag)
export STATEGRAPH_API_BASE=https://stategraph.example.com

stategraph tenant gaps config \
  --tenant=550e8400-e29b-41d4-a716-446655440000 \
  --provider=aws

Response:

{
  "status": "ready",
  "ready_for_gap_analysis": true,
  "ready_for_terraform_import": true,
  "has_aggregator": true,
  "aggregator_region": "us-east-1",
  "indexed_regions": ["us-east-1", "us-west-2", "eu-west-1"],
  "index_count": 3,
  "warnings": []
}

GCP

Gap analysis uses GCP Cloud Asset Inventory to discover resources.

Setup Requirements

1. Cloud Asset API must be enabled
2. IAM permissions for Cloud Asset Inventory
3. Scope access at project, folder, or organization level

Required IAM Permissions

For gap analysis (read inventory):

# Grant Cloud Asset Viewer role
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member='serviceAccount:SA_EMAIL' \
  --role='roles/cloudasset.viewer'

For generating Terraform configurations (optional):

# Grant Viewer role (read-only access to all resources)
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member='serviceAccount:SA_EMAIL' \
  --role='roles/viewer'

Enable Cloud Asset API

gcloud services enable cloudasset.googleapis.com --project=PROJECT_ID

Scope Configuration

GCP gap analysis can operate at different scopes:

If no scope is set, Stategraph auto-detects from Application Default Credentials.

Check Configuration

# Set API base (or use --api-base flag)
export STATEGRAPH_API_BASE=https://stategraph.example.com

stategraph tenant gaps config \
  --tenant=550e8400-e29b-41d4-a716-446655440000 \
  --provider=gcp

Response:

{
  "status": "ready",
  "ready_for_gap_analysis": true,
  "ready_for_terraform_import": false,
  "scope_type": "project",
  "scope_id": "projects/my-project",
  "has_org_access": false,
  "warnings": [
    {
      "code": "NO_VIEWER_PERMISSION",
      "message": "Optional: To generate Terraform configurations for unmanaged resources, grant 'roles/viewer' (read-only) to your service account.",
      "fix": "gcloud projects add-iam-policy-binding my-project \\\n  --member='serviceAccount:123456-compute@developer.gserviceaccount.com' \\\n  --role='roles/viewer'"
    }
  ]
}

Supported GCP Resource Types

72 resource types are supported, including:

Accessing Gap Analysis

Via UI

1. Navigate to Inventory → Gap Analysis
2. Select your cloud provider (AWS or GCP)
3. Click Check Config to verify setup
4. Click Analyze to run gap analysis
5. View unmanaged resources

The UI shows:

• Coverage Summary - Total resources, managed vs unmanaged
• Service Breakdown - Resources grouped by service
• Resource Table - Details of each unmanaged resource

Via CLI

Run gap analysis:

# Set API base (or use --api-base flag)
export STATEGRAPH_API_BASE=https://stategraph.example.com

stategraph tenant gaps analyze \
  --tenant=550e8400-e29b-41d4-a716-446655440000 \
  --provider=gcp

Response:

{
  "summary": {
    "total_cloud_resources": 500,
    "managed_by_stategraph": 450,
    "unmanaged": 50,
    "phantom_filtered": 12
  },
  "unmanaged_resources": [
    {
      "id": "//compute.googleapis.com/projects/my-project/zones/us-central1-a/instances/orphan-vm",
      "asset_name": "//compute.googleapis.com/projects/my-project/zones/us-central1-a/instances/orphan-vm",
      "asset_type": "compute.googleapis.com/Instance",
      "service": "compute",
      "region": "us-central1-a",
      "project_id": "my-project"
    }
  ],
  "fetched_at": 1705312800,
  "from_cache": false
}

Understanding Results

Summary Metrics

Phantom Resources

Some resources are automatically filtered because they're provider-managed and can't be imported:

AWS:

• Default Athena workgroups and catalogs
• Service-linked IAM roles
• Default event buses
• Default S3 Storage Lens dashboards

GCP:

• Default compute service accounts
• Default VPC networks
• Default firewall rules

Generating Terraform Configurations

Select unmanaged resources and generate import-ready Terraform code.

Via UI

1. Select resources using checkboxes
2. Click Generate Config
3. Review generated HCL
4. Copy or download the .tf file

Via CLI

# Set API base (or use --api-base flag)
export STATEGRAPH_API_BASE=https://stategraph.example.com

# Get unmanaged resources
stategraph tenant gaps analyze \
  --tenant=550e8400-e29b-41d4-a716-446655440000 \
  --provider=gcp | jq '.unmanaged_resources' > unmanaged.json

# Generate Terraform configuration
stategraph tenant gaps import \
  --tenant=550e8400-e29b-41d4-a716-446655440000 \
  --provider=gcp \
  unmanaged.json

Response:

{
  "generated_hcl": "resource \"google_compute_instance\" \"imported_orphan_vm\" {\n  name         = \"orphan-vm\"\n  machine_type = \"e2-medium\"\n  zone         = \"us-central1-a\"\n  ...\n}\n",
  "import_blocks": "import {\n  to = google_compute_instance.imported_orphan_vm\n  id = \"projects/my-project/zones/us-central1-a/instances/orphan-vm\"\n}\n",
  "provider_hcl": "terraform {\n  required_providers {\n    google = {\n      source  = \"hashicorp/google\"\n      version = \"~> 5.0\"\n    }\n  }\n}\n",
  "supported_count": 1,
  "unsupported_count": 0,
  "unsupported_resources": []
}

Using Generated Code

1. Save generated HCL to a .tf file
2. Run tofu plan or terraform plan to verify
3. Run tofu apply or terraform apply to import

# Save generated configuration
cat > import.tf << 'EOF'
import {
  to = google_compute_instance.imported_orphan_vm
  id = "projects/my-project/zones/us-central1-a/instances/orphan-vm"
}

resource "google_compute_instance" "imported_orphan_vm" {
  name         = "orphan-vm"
  machine_type = "e2-medium"
  zone         = "us-central1-a"
  # ... generated attributes
}
EOF

# Verify and import
tofu plan
tofu apply

Caching

Gap analysis results are cached to avoid rate limiting and reduce API calls:

# Set API base (or use --api-base flag)
export STATEGRAPH_API_BASE=https://stategraph.example.com

# Use cache (default, fast)
stategraph tenant gaps analyze \
  --tenant=550e8400-e29b-41d4-a716-446655440000 \
  --provider=gcp

# Force fresh scan (slower)
stategraph tenant gaps analyze \
  --tenant=550e8400-e29b-41d4-a716-446655440000 \
  --provider=gcp \
  --source=no-cache

The fetched_at timestamp and from_cache flag indicate data freshness.

In the UI, click Refresh to bypass the cache.

Filtering Results

By Service

stategraph tenant gaps analyze \
  --tenant=550e8400-e29b-41d4-a716-446655440000 \
  --provider=gcp | jq '.unmanaged_resources | map(select(.service == "compute"))'

By Region

stategraph tenant gaps analyze \
  --tenant=550e8400-e29b-41d4-a716-446655440000 \
  --provider=gcp | jq '.unmanaged_resources | map(select(.region == "us-central1-a"))'

By Resource Type

stategraph tenant gaps analyze \
  --tenant=550e8400-e29b-41d4-a716-446655440000 \
  --provider=gcp | jq '.unmanaged_resources | map(select(.asset_type == "compute.googleapis.com/Instance"))'

Troubleshooting

AWS: No indexes found

"warnings": [{"code": "NO_INDEXES", "message": "No Resource Explorer indexes found"}]

Solution: Enable Resource Explorer in at least one region:

aws resource-explorer-2 create-index --type LOCAL

AWS: No aggregator

"warnings": [{"code": "NO_AGGREGATOR", "message": "No aggregator index configured"}]

Solution: Upgrade one index to aggregator type for cross-region discovery:

aws resource-explorer-2 update-index-type --arn "INDEX_ARN" --type AGGREGATOR

GCP: Permission denied

"warnings": [{"code": "PERMISSION_DENIED", "message": "Permission denied for scope projects/my-project"}]

Solution: Grant Cloud Asset Viewer role:

gcloud projects add-iam-policy-binding my-project \
  --member='serviceAccount:YOUR_SA@my-project.iam.gserviceaccount.com' \
  --role='roles/cloudasset.viewer'

GCP: Insufficient OAuth scopes

"warnings": [{"code": "INSUFFICIENT_SCOPES", "message": "VM OAuth scopes are insufficient"}]

Solution: Update VM scopes (requires restart):

gcloud compute instances stop INSTANCE_NAME --zone=ZONE
gcloud compute instances set-service-account INSTANCE_NAME \
  --zone=ZONE --scopes=cloud-platform
gcloud compute instances start INSTANCE_NAME --zone=ZONE

Terraform import fails with permission denied

The gap analysis uses Cloud Asset API, but Terraform import needs actual resource read permissions.

Solution: Grant roles/viewer for Terraform operations:

gcloud projects add-iam-policy-binding my-project \
  --member='serviceAccount:YOUR_SA@my-project.iam.gserviceaccount.com' \
  --role='roles/viewer'

Best Practices

1. Check config first - Run config check before analysis to verify permissions
2. Regular scans - Run gap analysis periodically (weekly or after deployments)
3. Triage results - Classify gaps as intentional or concerning
4. Import systematically - Bring unmanaged production resources under control
5. Use organization scope - For GCP, use organization-level access for complete visibility
6. Document exceptions - Record why some resources remain unmanaged

Environment Variables

Credential Configuration

This section explains how to pass cloud provider credentials to Stategraph for gap analysis. By default, Stategraph uses the standard credential chain for each provider (instance metadata, environment variables, config files).

AWS Credentials

Default Behavior

Stategraph uses the standard AWS credential chain:

1. Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
2. Shared credentials file (~/.aws/credentials)
3. IAM role for Amazon EC2 / ECS task role
4. IAM Roles for Service Accounts (IRSA) in EKS

Docker Compose

Option 1: Environment Variables

Pass credentials directly (suitable for development):

services:
  server:
    image: ghcr.io/stategraph/stategraph-server:latest
    environment:
      # ... existing config ...
      AWS_ACCESS_KEY_ID: "AKIAIOSFODNN7EXAMPLE"
      AWS_SECRET_ACCESS_KEY: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
      AWS_DEFAULT_REGION: "us-east-1"
      # Optional: for temporary credentials
      # AWS_SESSION_TOKEN: "your-session-token"

Option 2: Mount AWS Config Directory

Mount your local AWS credentials (recommended for development):

services:
  server:
    image: ghcr.io/stategraph/stategraph-server:latest
    environment:
      # ... existing config ...
      AWS_DEFAULT_REGION: "us-east-1"
      # Optional: specify a named profile
      # AWS_PROFILE: "my-profile"
    volumes:
      - ~/.aws:/home/stategraph/.aws:ro

Option 3: Use .env File

Store credentials in a .env file (not committed to git):

# .env
AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
AWS_DEFAULT_REGION=us-east-1

services:
  server:
    image: ghcr.io/stategraph/stategraph-server:latest
    env_file:
      - .env

Kubernetes

Option 1: Secret with Environment Variables

apiVersion: v1
kind: Secret
metadata:
  name: aws-credentials
  namespace: stategraph
stringData:
  AWS_ACCESS_KEY_ID: "AKIAIOSFODNN7EXAMPLE"
  AWS_SECRET_ACCESS_KEY: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: stategraph
  namespace: stategraph
spec:
  template:
    spec:
      containers:
        - name: stategraph
          envFrom:
            - secretRef:
                name: aws-credentials
          env:
            - name: AWS_DEFAULT_REGION
              value: "us-east-1"

Option 2: IAM Roles for Service Accounts (IRSA) - Recommended for EKS

1. Create an IAM role with the required permissions and trust policy for your EKS cluster
2. Annotate the Kubernetes service account:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: stategraph
  namespace: stategraph
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/stategraph-gap-analysis
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: stategraph
  namespace: stategraph
spec:
  template:
    spec:
      serviceAccountName: stategraph
      containers:
        - name: stategraph
          env:
            - name: AWS_DEFAULT_REGION
              value: "us-east-1"

Option 3: EC2 Instance Profile

If running on EC2 or using node IAM roles, no additional configuration is needed. Ensure the node's IAM role has the required Resource Explorer permissions.

GCP Credentials

Default Behavior

Stategraph uses the standard GCP credential chain:

1. GOOGLE_APPLICATION_CREDENTIALS environment variable (path to service account JSON)
2. Application Default Credentials (ADC)
3. Compute Engine / GKE metadata server
4. Workload Identity (GKE)

Docker Compose

Mount your GCP service account key file and set GOOGLE_APPLICATION_CREDENTIALS to point to it:

services:
  server:
    image: ghcr.io/stategraph/stategraph-server:latest
    environment:
      # ... existing config ...
      GOOGLE_APPLICATION_CREDENTIALS: "/secrets/gcp-sa-key.json"
      GOOGLE_CLOUD_PROJECT: "my-project-id"
    volumes:
      - ./gcp-sa-key.json:/secrets/gcp-sa-key.json:ro

This approach:
- Uses the standard GCP authentication method via GOOGLE_APPLICATION_CREDENTIALS
- Mounts your service account key file into the container
- Works consistently across Docker Compose and Kubernetes environments

Note: Ensure your service account key file is excluded from version control (add to .gitignore)

Kubernetes

Option 1: Secret with Service Account Key

1. Create a secret from your service account JSON:

kubectl create secret generic gcp-credentials \
  --namespace=stategraph \
  --from-file=key.json=./gcp-sa-key.json

2. Mount the secret and set the environment variable:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: stategraph
  namespace: stategraph
spec:
  template:
    spec:
      containers:
        - name: stategraph
          env:
            - name: GOOGLE_APPLICATION_CREDENTIALS
              value: "/secrets/gcp/key.json"
            - name: GOOGLE_CLOUD_PROJECT
              value: "my-project-id"
          volumeMounts:
            - name: gcp-credentials
              mountPath: /secrets/gcp
              readOnly: true
      volumes:
        - name: gcp-credentials
          secret:
            secretName: gcp-credentials

Option 2: Workload Identity (Recommended for GKE)

1. Enable Workload Identity on your GKE cluster
2. Create a GCP service account with the required permissions
3. Bind the Kubernetes service account to the GCP service account:

gcloud iam service-accounts add-iam-policy-binding \
  stategraph-sa@PROJECT_ID.iam.gserviceaccount.com \
  --role="roles/iam.workloadIdentityUser" \
  --member="serviceAccount:PROJECT_ID.svc.id.goog[stategraph/stategraph]"

4. Annotate the Kubernetes service account:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: stategraph
  namespace: stategraph
  annotations:
    iam.gke.io/gcp-service-account: stategraph-sa@PROJECT_ID.iam.gserviceaccount.com
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: stategraph
  namespace: stategraph
spec:
  template:
    spec:
      serviceAccountName: stategraph
      containers:
        - name: stategraph
          env:
            - name: GOOGLE_CLOUD_PROJECT
              value: "my-project-id"

Option 3: GCE Metadata Server

If running on GCE or GKE without Workload Identity, the default service account is used automatically. Ensure it has the required Cloud Asset Viewer permissions.

Multi-Cloud Configuration

To enable gap analysis for both AWS and GCP simultaneously:

Docker Compose:

services:
  server:
    image: ghcr.io/stategraph/stategraph-server:latest
    environment:
      # AWS
      AWS_DEFAULT_REGION: "us-east-1"
      # GCP
      GOOGLE_APPLICATION_CREDENTIALS: "/secrets/gcp-sa-key.json"
      GOOGLE_CLOUD_PROJECT: "my-project-id"
    volumes:
      - ~/.aws:/home/stategraph/.aws:ro
      - ./gcp-sa-key.json:/secrets/gcp-sa-key.json:ro

Kubernetes:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: stategraph
  namespace: stategraph
spec:
  template:
    spec:
      serviceAccountName: stategraph  # With IRSA and/or Workload Identity
      containers:
        - name: stategraph
          env:
            # AWS
            - name: AWS_DEFAULT_REGION
              value: "us-east-1"
            # GCP
            - name: GOOGLE_APPLICATION_CREDENTIALS
              value: "/secrets/gcp/key.json"
            - name: GOOGLE_CLOUD_PROJECT
              value: "my-project-id"
          volumeMounts:
            - name: gcp-credentials
              mountPath: /secrets/gcp
              readOnly: true
      volumes:
        - name: gcp-credentials
          secret:
            secretName: gcp-credentials

Next Steps

• Dashboards - Create custom views of your inventory
• Query Language - Query resources with MQL
• Environment Variables - Full configuration reference