Guides
Log InSign UpBlog

Storage Profiles

Suggest Edits

Storage profiles enable organizations to customize how and where blobs are stored. A blob is Terraform/OpenTofu state files, code, and run artifacts such as logs. Profiles can be set via the API and viewed by going to the account scope, then security, and storage profiles:

Storage Profiles Options

Scalr storage profiles provide two primary deployment models:

  • Scalr-Managed Storage (Default): Blobs stored in a Scalr-managed bucket. This is the default for all accounts.
  • Customer-Managed Storage: Blobs stored in your own AWS S3 or GCP buckets.

Customer-Managed Blob Storage

You can store state and other blob objects in your GCP or AWS S3 bucket with storage profiles.
This option is ideal when your organization requires state to be stored in your infrastructure
or you have a specific requirement about data residency.

The following objects will be stored in your bucket using customer-managed storage:

  • State files
  • Terraform/OpenTofu Configuration Versions
  • Plan JSON and binaries
  • Terraform/OpenTofu logs

GCP Storage Profile Configuration

To configure a GCP bucket storage profile, first create a bucket in GCP. The bucket must be accessible by Scalr and have proper permissions set. Once the bucket is created and accessible, make the API call to Scalr to create the storage profile. The following must be specified in the API call:

  • GCP Account ID
  • GCP service account JSON key with the IAM role Storage Admin assigned on a google-storage-bucket
  • GCP encryption key (optional)
  • GCP project ID
  • GCP storage bucket name

Sample GCP bucket configuration:

Location type:         Multi-region
Default storage class: Standard
Public access:         Subject to object ACLs
Access control:        Fine-grained
Protection:            Soft Delete
Bucket retention:      None
Lifecycle rules:       None
Encryption:            Google-managed

Call the Scalr API to set the new profile. This can be done through he interactive API docs.

Example API request to Scalr to create a GCP storage profile:

POST {{host}}/api/iacp/v3/storage-profiles
Authorization: Bearer {{token}}
Content-Type: application/vnd.api+json

{
  "data": {
    "type": "storage-profiles",
    "attributes": {
      "backend-type": "google",
      "default": true,
      "name": "gcp-ape-storage-profile",
      "google-storage-bucket": "gcp-bucket-playground",
      "google-project": "scalr-dev",
      "google-credentials": {
          GCP service account JSON key
      }

    }
  }
}

AWS S3 Storage Profile Configuration

To configure an AWS S3 storage profile, you must create the bucket, set up OIDC authentication, and link an IAM role:

  1. Create an S3 bucket in AWS. The bucket must be accessible by Scalr.
  2. Set up AWS OIDC authentication:
    • In AWS IAM, add an identity provider with URL https://scalr.io
    • Create an appropriate audience value.
  3. Create an AWS IAM role:
    • Select web identity and your OIDC provider.
    • Assign AmazonS3FullAccess permission or create a custom policy.
    • For KMS encryption, add permissions:
      • kms:GenerateDataKey
      • kms:Decrypt

After the role is created, an additional assertion can be added to limit which environments can use the role. Follow these steps to add the assertion:

  1. Open the Trust Relationships tab in the IAM role dashboard and click the Edit trust policy button.
  2. In the condition section, add the following snippet:
"StringEquals": {
    "scalr.io:aud": "my-awesome-audience"
},
"StringLike": {
  "scalr.io:sub": "account:<account-name>"
}

Here's a full example of the trust policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowScalrOIDCAccess",
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::111111111:oidc-provider/scalr.io"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "scalr.io:aud": "my-awesome-audience"
                },
                "StringLike": {
                    "scalr.io:sub": "scalr:account:example"
                }
            }
        }
    ]
}

When creating the storage profile in Scalr, the following must be specified in the API callv:

  • Backend Type: aws-s3
  • AWS S3 Audience: Value provided during OIDC provider creation
  • AWS S3 Bucket Name: Name of your S3 bucket
  • AWS S3 Role ARN: ARN of the created IAM role
  • AWS Region: Region where your bucket is located

Example API request to create an AWS S3 storage profile:

POST {{host}}/api/iacp/v3/storage-profiles
Authorization: Bearer {{token}}
Content-Type: application/vnd.api+json

{
  "data": {
    "type": "storage-profiles",
    "attributes": {
      "backend-type": "aws-s3",
      "aws-s3-bucket-name": "example-aws-bucket",
      "aws-s3-audience": "my-awesome-audience",
      "aws-s3-region": "us-east-1",
      "aws-s3-role-arn": "arn:aws:iam::123456789012:role/your-role-name",
      "default": true,
      "name": "aws-s3-storage-profile"
    }
  }
}

The list of all AWS OIDC condition context keys:

  • aud is {audience_from_resource}
  • sub is scalr:account:{acc_name}
  • scalr_account_id is {acc_id}
  • scalr_account_name is {acc_name}

Limitations

  • After a blob object is created in a profile, only the name of the storage profile can be updated; no other field can be changed.
  • Currently, it's not possible to delete a storage profile if blob objects exist. All objects must be cleaned up before it can be deleted.
  • If you need to store new blobs in a different profile, you can create a new one and mark it as the default. All NEW blob objects will be stored in the new profile, existing blob objects will NOT be migrated.

Storage Profile at Environment Level (Coming Soon)

We are developing the capability to create storage profiles at the environment level, where all run artifacts can be saved in a separate storage profile attached to the environment.

Updated about 19 hours ago