Drift Detection

👍

Drift detection runs do NOT count toward billing.

Overview

Terraform and OpenTofu drift occur when the actual state of your infrastructure differs from the state defined in your configuration files. It's caused by changes made to resources outside of the standard Terraform workflow, such as a manual edit in a cloud provider's console. These out-of-band changes cause the Terraform state file to become an inaccurate representation of your infrastructure.

Configuration

In Scalr, drift detection can be enabled per environment. If enabled in an environment, all workspaces in the environment will have the detection schedule applied.

To enable it, go to the environment management settings, click on drift detection, and apply the schedule:

Once enabled, the drift detector will execute based on the schedule. If drift is detected, workspace owners will see the drifted run in the drift detection tab. Drift detection runs do not block the run queue if drift is found and waiting on your decision (Sync state or revert infrastructure).

Drift will execute if the workspace has the following:

  • The workspace has active state.
  • A workspace has not been applied during the configured period: daily - in the last 24 hours, weekly - in the last 7 days.
  • Remote execution is mode enabled.

Drift Remediation

When reviewing drift within a workspace, a user will have three options to remediate the drift:

  • Ignore: Changes found by the drift detector are declined, and no further actions are performed. Users can choose to resolve the drift manually.
  • Sync State: Changes found by the drift detector will be written to a state file. After clicking "Sync State", a refresh-only run will be triggered to synchronize the state. The runs:create permission is required.
  • Revert Infrastructure: Changes found by the drift detector will be declined. After clicking "Revert Infra", a plan & apply run will be triggered to rollback infrastructure to the previous state. The runs:create permission is required.

Runs executed to sync state or revert infrastructure will count towards billing.

Drift Notifications

You can integrate Scalr with Slack to receive real-time notifications upon the detection of infrastructure drift. By directing these alerts to a specific Slack channel, you ensure the appropriate team members are promptly informed. See more on configuring this here.

Once it is configured, the notifications will appear in Slack and then the actions can also be taken directly from there:

Other tools, such as MS Teams, will have drift detection added soon.

Drift Reporting

Operational dashboards provide drift reporting capabilities at both the account and environment levels. Use the built-in filters to create a view that shows only workspaces with detected drift. From this dashboard, you can select any drifted workspace to investigate its details and take remediation action.

This can be done in the workspace dashboard:

Or within the stale workspace report. The stale workspace report helps identify workspaces that have not had runs executed on them in a certain time frame, which means there is likely a greater chance that drift has happened:

Demo

More of a visual learner? Check out the demo here:

See how all of these drift notification features tied together here: https://scalr.com/learning-center/terraform-drift-detection-how-to-prevent-and-remediate/