Introduction

This documentation outlines the support for the OpenMetrics API and guides integrating it with Datadog or Splunk. The OpenMetrics API is a vendor-neutral, open standard for transmitting metrics at scale, primarily focused on observability for cloud-native environments. Integrating the OpenMetrics API with monitoring and logging platforms like Datadog and Splunk enables organizations to collect, analyze, and visualize metrics effectively, facilitating better insights into system performance and behavior.

The Scalr endpoint for seeing account metrics is: https://docs.scalr.io/reference/get_open_metrics

Scalr will expose the following information for consumption:

# TYPE scalr_environments_count gauge
scalr_environments_count 10
# TYPE scalr_workspaces_count gauge
scalr_workspaces_count 137
# TYPE scalr_quota_max_concurrency gauge
scalr_quota_max_concurrency 5
# TYPE scalr_runs_count gauge
scalr_runs_count 1225
# TYPE scalr_runs_awaiting_confirmation gauge
scalr_runs_awaiting_confirmation 14
# TYPE scalr_runs_failed gauge
scalr_runs_failed 299
# TYPE scalr_runs_successful gauge
scalr_runs_successful 733
# TYPE scalr_runs_concurrency gauge
scalr_runs_concurrency 3
# TYPE scalr_runs_queue_size gauge
scalr_runs_queue_size 0
# TYPE scalr_billings_runs_count gauge
scalr_billings_runs_count 1000
# TYPE scalr_billings_run_minutes_count gauge
scalr_billings_run_minutes_count 2000
# TYPE scalr_billings_flex_runs_count gauge
scalr_billings_flex_runs_count 0
# TYPE scalr_billings_flex_runs_minutes_count gauge
scalr_billings_flex_runs_minutes_count 0
# EOF

Example Platforms

The OpenMetrics API can be integrated with various monitoring and logging platforms. In this documentation, we will focus on integration with two popular platforms, but the integration is not limited to these:

1. Datadog: A cloud-scale monitoring and analytics platform that provides comprehensive observability solutions for cloud, microservices, and containerized environments.
2. Splunk: A leading platform for collecting, analyzing, and visualizing machine data, including logs, metrics, and security events.

Integration Steps

Datadog Integration

Step 1: Configure Datadog Agent

• Install and configure the Datadog Agent on any server or container, this does not have to be Scalr specific.
• Edit the Datadog Agent configuration file (datadog.yaml) to include the OpenMetrics endpoint as a custom check. Replace the account-name placeholder with your account name:

   instances:  
      openmetrics_endpoint: https\://<account-name>.scalr.io/api/iacp/v3/metrics  
        metrics:  
            - .\*  
          headers:  
            Authorization: Bearer <token>  
            Prefer: profile=preview
  

• Restart the Datadog Agent to apply the configuration changes.

Step 2: Verify Integration

Verify that Datadog receives metrics from the OpenMetrics endpoint by checking the Datadog dashboard.

More information about the integration with Datadog read here: Basic Agent Usage for Ubuntu and the OpenMetrics Integration.

Splunk Integration

Step 1: Install Splunk HTTP Event Collector (HEC)

• Install and configure the Splunk HTTP Event Collector (HEC) on your Splunk instance.
• Note down the HEC token provided during the HEC setup process

Here is an example configuration, replace the account-name placeholder with your account name:

receivers:  
  smartagent/prometheus-exporter:  
    type: prometheus-exporter  
    extraDimensions:  
      metric_source: prometheus  
    useHTTPS\: true  
    httpHeaders:  
      authorization: "Bearer <token>"  
      prefer: profile=internal  
    host: "<account-name>.scalr.io"  
    port: 443  
    metricPath: "/api/iacp/v3/metrics"

Step 2: Configure Data Inputs in Splunk

• Log in to your Splunk instance and navigate to Settings > Data Inputs.
• Select HTTP Event Collector and create a new HEC data input.
• Configure the data input with the appropriate settings, including the HEC token and source type.

Example:

service:  
  pipelines:  
    metrics:  
      receivers: [smartagent/prometheus-exporter]

Step 3: Send Metrics to Splunk

In your application, use a library or tool that supports sending metrics to Splunk via HTTP Event Collector. You may include the HEC token and configure the appropriate endpoint URL.

Step 4: Verify Integration

Verify that Splunk is receiving metrics from the OpenMetrics endpoint by searching for the data in the Splunk interface.

More information about the integration you can find here:

• Install the Collector for Linux with the installer script — Splunk Observability Cloud documentation
• Prometheus Exporter — Splunk Observability Cloud documentation

Summary

As mentioned, the above examples are just two of the platforms that support OpenMetrics. The majority of monitoring an analytics platforms support the ability to ingest data from OpenMetrics. Please review the documentation for your monitoring and analytics tool to see how to integrate it with Scalr.