Runs

A run is the result of executing a Terraform or OpenTofu deployment in a workspace. There are two types of runs, runs that include an apply and runs that exclude an apply, referred to as a dry run. All runs for a given workspace will be displayed in the runs tab. For VCS-driven runs, a commit hash is provided, which can be clicked on to help users understand what changes were made to the code prior to the deployment for the entire history of the workspace. CLI runs will be noted as CLI-driven with the username of the person who created the run. No matter what the entry point for a run is, they will follow this workflow:

Run Stage	Description	Required
Pre-Init	The pre-init stage is often used to install binaries that are needed during the run. A custom hook is the only option during pre-init.	No
Terraform/Tofu Init	The init stage initializes providers, modules, plugins, and working directories that will be used during the run.	Yes
Pre-Plan	The pre-plan stage can be used to do the following: - Run `[terraform fmt](https://docs.scalr.io/docs/workspaces-4#custom-hooks)` to check the formatting of the code. - Run `[terraform validate](https://docs.scalr.io/docs/workspaces-4#custom-hooks)` to check the syntax of the code. - Run a custom hook to execute a script or command within the run time container. - Run OPA based on the `tf-run`input. - Run Checkov.	No
Terraform/Tofu Plan	The plan stage executes a run that shows you what changes will happen to the resources based on the configuration files.	Yes
Post-Plan	The post-plan stage can be used for the following: - Generating a cost estimate using the Infracost integration. - Run policy checks via OPA against the plan JSON file, which includes the `tfrun` and `tfplan` details. - Run a custom hook to execute a script or command within the run time container, such as exporting the plan details.	No
OPA Approval	If an OPA policy is enforced with the `soft-mandtory` enforcement level, the run will be stopped for approval if the plan violates the given policy. Users with the `policy-checks:override`permission will be able to approve this stage.	No
Apply Approval	If the workspace settings has `auto-apply` disabled, then all runs for the workspace will wait for an approval before executing an `apply`. Users with `runs:apply` will be able to approve this stage. Approvals can also be done through Slack or Microsoft Teams.	No
Pre-Apply	The pre-apply stage is often used to install binaries, call an external API, or any other requirements that are needed during the run. A custom hook is the only option during pre-apply.	No
Terraform/Tofu Apply	The apply stage creates the resources that were shown in the plan stage.	Yes
Post Apply	The post-apply stage is often used to call an external API to notify of the run results, download the state externally, or any other requirement needed after the apply. A custom hook is the only option during pre-apply.	No

Dry Runs (Plan-Only)

A dry run is when a Terraform or OpenTofu plan is executed. Dry runs will check the state file against the resources in the provider and show the proposed changes, if any. If there are changes, the run will go through the cost-estimation and policy check phase. If there are no changes, the run will stop after the plan with a "no changes" status.

Dry runs can be executed using the Terraform or Tofu CLI if the remote backend is added locally to the workspace. Dry runs can also be executed if a pull request is opened in a repository that is linked to a Scalr workspace. The workspace must have the proper trigger strategy set for Scalr to recognize the changes and start a run. Depending on the VCS provider you are using, Scalr will send information back to the pull request comments for the teams to review.

Pull Request Comments

If a workspace source is VCS, Scalr will execute a dry run based on a pull request and report the plan as well as the apply result back to the VCS provider as a pull request comment.

Github, Gitlab, and Azure DevOps have a slightly advanced output in that you can see an overall summary for the pull request as well as a per workspace report (click on the VCS links in the beginning of this paragraph to learn how to enable this). If everything is successful, Scalr will return the success status and you are provided with a general summary. If there are destructive changes, resources being replaced, or policy failures, you will see a warning. For any Terraform or OpenTofu runs that fail, the error logs will be sent back into the PR comments.

Scalr will execute an apply when the pull request is merged and the apply results will also be posted back into the same pull request as a comment.

With Bitbucket, the checks are more abbreviated in that you do not have full pull request comments. The "main" check seen below is a cumulative check across all workspaces that are connected to this repo and the working directory (if applicable). This check can be used to create a branch policy to ensure only successful dry runs are merged. There will only be a successful green check if all runs are successful. These

Execute Runs from Pull Request Comments

For VCS-based workspaces, a plan or apply run can be executed directly from pull request comment. As of right now, only Github is supported, but other VCS providers will be added soon. To execute a run from a comment simply execute the following in a pull request comment:

/scalr plan

/scalr apply

The status of the run will be sent back to the VCS provider as a comment:

If the workspace setting "Enabled VCS-driven dry runs" is disabled, the /scalr plan command will not work, but apply will. To override this, use the -force flag to force a plan to execute: /scalr plan -force

Command List:

Command	Description
`/scalr plan`	Executes a dry run in Scalr and results are returned to the VCS provider.
`/scalr plan -force`	Forces a dry runs to execute even if the "Enabled VCS-driven dry runs" is disabled in the workspace settings.
`/scalr apply`	Executes an apply run in Scalr and results are returned to the VCS provider.
`/scalr apply -workspace-id=<workspace-id>`	Executes an apply run targeted to a specific workspace and results are returned to the VCS provider. Useful for repositories linked to many workspaces.

The ability to execute runs from a comment is controlled through the [VCS provider settings](https://docs.scalr.io/docs/github).

Apply Runs

An apply run is a run in which the Terraform or Tofu apply command is executed and resources are created based on a successful run. The apply run executes the plan first to make sure there are changes. If there are changes it then goes through the cost estimation, policy check, and then executes the apply to create the resources. An apply run can be executed through the UI, from the Terraform or Tofu CLI, or by merging a pull request if the workspace is linked to a VCS repository:

Drift

Drift happens when the actual state of a resource does not match with what is expected in the Terraform configuration file. Scalr will automatically detect drift and show it in the run console as well as the run dashboard:

Scalr will not auto remediate the drift, instead it us up to the user reviewing the run to determine how to handle the drift.

Schedule Drift

Drift can be determined by executing a run in refresh-only. When running refresh-only, Terraform will check the state file vs the actual state of the infrastructure and show the differences, if any. If drift is detected, it will be displayed and the run will go into an approval step. If approved, an apply will execute to update the state file to match what the actual state of the infrastructure.

Pre-Checks

Stop wasting time finding issues after a run has executed. The Terraform and OpenTofu pre-check feature allows you to validate code before the run is executed. These can be added to all workspaces by enabling them in the Terraform or OpenTofu integration located at the account scope -> integrations section. Any run that fails during a pre-check is not counted towards billing.

The following checks are available:

terraform fmt
terraform validate
opentofu fmt
opentofu validate

Run Modes

When queuing a new run from the UI, you will have the options below presented to you. All standard Terraform or OpenTofu commands are still available when executing from the CLI.

Plan & Apply - The default option. Initiates a full cycle of the plan, apply, (cost and policy checks are executed only if are configured in the environment).
Plan Only - Only execute a terraform plan, which includes the cost-estimate and policy checks.
Refresh-only - A new configuration is not applied, Terraform only updates the state according to the actual value in the 3rd party.
Skip Refresh - The current state is not refreshed while planning the changes.
Replace - A limited list of resources can be recreated. At least 1 resource is required to be selected.
Target Resources - A list of resources that will be targeted during the plan and apply. All other resources will be ignored.

Scheduled Applies

Want to approve a Terraform or OpenTofu apply, but delay when it actually executes? Scheduled applies allow you to set a one time schedule for the apply that is being approved:

Run Dashboards

The runs dashboard, which exists at the account, environment, and workspace scope, serves as a central dashboard for all runs across all workspaces. From this page, runs can be canceled in bulk or approved/discarded as needed. A use case for the bulk cancellation is to reprioritize runs (i.e. you have an emergency change going in that cannot wait on prior runs to finish).

Click through demo:

The permissions to view the runs page can be controlled through theruns-queue:read in the IAM roles.

Run Output

The output of a plan can be displayed in two different ways, the standard console output or through the smart plan output referred to as a visual plan.

The visual plan is useful when you have hundreds or thousands of resources and you need to quickly filter on the action that is occurring in the plan:

Ignore Draft Pull Requests

Draft pull requests are commonly used during the early development cycles to note when a pull request is in a draft state. Since draft pull requests are so early on in the development cycle, it's possible that you don't want a Scalr plan to execute when pull requests are opened. Scalr gives you the option to ignore draft pull requests by unchecking "Trigger runs for draft pull requests". As of right now, this will apply to any workspace that is linked to the VCS provider.

Bulk Runs

Need to kick off runs in multiple workspaces at the same time? Bulk actions allow you to multi-select workspaces at the account or environment scope to execute a run with any of the run modes seen above or a destroy.

Prioritize Runs

During active development phases, runs will build up in the queue. It's much easier to select the run you want to actually execute rather than having to go through and delete other runs one by one, which is what the force run feature does. By forcing a run, Scalr will automatically discard all previous pending runs to ensure the selected run executes as fast as possible. Just click on the lightning bolt to select the run to execute.

Target Resources

The target option gives users the ability to focus the Terraform run on a specific resource or set of resources:

After a target is checked, select one or more resources that the run will impact once executed.

OIDC connect

Overview

Scalr has built-in support for major cloud providers (AWS, Azure, Google) via Provider configurations. However, some custom providers (Cloudflare, Vault, etc.) don't have the OIDC-connected options via the provider configurations. To solve this issue, Scalr exports an environment variable, SCALR_OIDC_TOKEN, for each run phase (plan & apply) that can be used to connect to the custom provider.

Token payload

The OIDC token is signed by the Scalr JWT token and can be exchanged for a temporary token by the custom provider. The payload has the following structure:

{
  "aud": "<account-name>",
  "exp": 1700056966,
  "iat": 1700053066,
  "iss": "https://scalr.io",
  "jti": "0ddab61b-4ca3-4112-be31-7549c83f26b8",
  "nbf": 1700053066,
  "ref": "",
  "scalr_account_id": "acc-xxxxxxxx",
  "scalr_account_name": "<account-name>",
  "scalr_environment_id": "env-xxxxxxxx",
  "scalr_environment_name": "<environment-name>",
  "scalr_run_id": "run-xxxxxxx",
  "scalr_run_phase": "plan",
  "scalr_tags": "",
  "scalr_workspace_id": "ws-xxxxxxxxxxxxx",
  "scalr_workspace_name": "<workspace-name>",
  "sub": "account:<account-name>:environment:<environment-name>:workspace:<workspace-name>"
}

Issuer always equals to https://scalr.io.

Payload customization

To make the token generation more streamlined, Scalr adds default values for the following attributes:

aud - by default, it is the account name. To override it, you can create the shell variable SCALR_OIDC_AUD at any level and it will be set for all workspaces within the scope it is created (workspace, environment, or account).
exp - by default, it is set to the run operation timeout of the workspace. If the timeout is not set, then the system's default value is used (60 minutes). To override it, you can create the shell variable SCALR_OIDC_EXP at any level and it will be set for all workspaces within the scope it is created (workspace, environment, or account).

Built-In Variables

The following shell variables are built into the runtime environment for use as needed:

SCALR_RUN_ID - The ID of the current run.
SCALR_HOSTNAME - The Scalr hostname.
SCALR_TERRAFORM_OPERATION - The current Terraform operation (plan or apply).
SCALR_TERRAFORM_EXIT_CODE - The exit code (0 or 1) of the previous operation (plan or apply) and only available in after hooks.
SCALR_RUN_IS_DESTROY - If the code is 1, then the run is the equivalent to a terraform destroy. If it is 0, then it is a terraform apply. This is helpful for custom hooks that will execute if the apply is a destroy or not.
SCALR_RUN_VCS_BRANCH - The branch name of the VCS repo i.e. main
SCALR_RUN_VCS_COMMIT- The commit hash for a VCS triggered run i.e. e59f09e
SCALR_RUN_MODE - This will return the type of run that was executed. Possible values are:
- apply - Plan & Apply
- dry - Plan only
- refresh - Refresh only
- skip_refresh - Skip refresh
SCALR_RUN_SOURCE - The source that triggered the run. Possible values are:
- vcs - The run was triggered by a VCS provider.
- cli - The run was triggered from the Terraform or OpenTofu CLI.
- scalr-cli- The run was triggered from the Scalr CLI
- run-trigger - The run was triggered from the run triggers feature.
- schedule - The run was triggered based on a schedule being set in the workspace.
- api- The run was triggered from the Scalr API.
- configuration version - When the run is triggered based on auto-queue runs being enabled in the workspace and the first CV uploaded triggers a run.
- ui - for all runs triggered from the UI.
SCALR_WORKSPACE_NAME- The name of the workspace the run is executing in.
SCALR_WORKSPACE_ID - The ID of the workspace the run is executing in.
SCALR_ENVIRONMENT_ID- The ID of the environment the run is executing in.
SCALR_ENVIRONMENT_NAME- The name of the environment the run is executing in.
SCALR_OIDC_TOKEN - The ID token that is exported for each run phase (plan & apply) that allows users to connect to third parties (e.g. Vault, Cloudflare, etc.) via the OIDC. More about OIDC connect read here

See the full documentation for variables here: Variables