Skip to main content

Deploying Infrastructure Updates with Docker Tags

Quick Reference​

# Set custom image tag
export DATA_LAKE_ETL_IMAGE_TAG=sha-XXXXX

# Preview changes
NX_TUI=false pnpm nx pulumi data-lake-infra -- preview --stack dev

# Deploy
NX_TUI=false pnpm nx pulumi data-lake-infra -- up --stack dev

Finding Docker Tags from GitHub Actions​

Location: [PLACEHOLDER: GitHub Actions workflow URL pattern]

  1. Navigate to GitHub Actions → Latest successful workflow run of you PR
  2. Find the Build & Pack Stage
  3. Find the package step output for the specific stack e.g. data-lake-ingest:pack
  4. Look for docker tag step
  5. Extract release field value (format: sha-{short-commit})

Deploying with Custom Tags​

Environment Variable Format​

{REPO_NAME}_IMAGE_TAG=sha-{commit}

Examples:

  • DATA_LAKE_ETL_IMAGE_TAG=sha-8dc9e82
  • DATA_LAKE_COLLECTOR_IMAGE_TAG=sha-829d5d5
  • DATA_WAREHOUSE_INGEST_IMAGE_TAG=sha-{commit}

Tag Resolution Priority​

  1. Custom repo tag: {REPO_NAME}_IMAGE_TAG (highest priority)
  2. Global override: IMAGE_TAG

See: packages/shared/infra/infra-core/src/lib/image.ts:139-208

Common Projects​

ProjectRepositoryExample Tag Var
data-lake-infradata-lake-etlDATA_LAKE_ETL_IMAGE_TAG
data-lake-infradata-lake-collectorDATA_LAKE_COLLECTOR_IMAGE_TAG
data-lake-infradata-lake-orchestrationDATA_LAKE_ORCHESTRATION_IMAGE_TAG
data-warehouse-infradata-warehouse-ingestDATA_WAREHOUSE_INGEST_IMAGE_TAG

Troubleshooting​

Image Tag Not Found​

Error: Image with tag sha-xxx not found in repository

Solution:

  1. Verify tag exists in ECR: aws ecr describe-images --repository-name cartesianio/{repo-name} --image-ids imageTag=sha-xxx
  2. Check GitHub Actions workflow completed successfully
  3. Ensure correct commit SHA format

Tag Resolution Issues​

Check tag resolution order:

  • Pulumi logs show [CUSTOM], [OVERRIDE], [RELEASE], or [DEPLOYED]
  • Verify environment variable is set: echo $DATA_LAKE_ETL_IMAGE_TAG
  • Check build-info.json exists: cat packages/data-lake/data-lake-etl/dist/build-info.json

References​

  • Tag resolution logic: packages/shared/infra/infra-core/src/lib/image.ts
  • Build info structure: packages/data-lake/data-lake-etl/dist/build-info.json
  • ECR registry: [PLACEHOLDER: ECR registry URL]
  • Pulumi console: https://app.pulumi.com/cartesianio/{project}/{stack}