Edit

Share via

Send data to an Azure Monitor workspace from your Prometheus server

  • Article

Azure Red Hat OpenShift is preinstalled with a default Prometheus server. As detailed in the Azure Red Hat OpenShift support policy, this default Prometheus server shouldn't be removed.

In some scenarios, you might want to centralize data from self-managed Prometheus clusters for long-term data retention to create a centralized view across your clusters. You can use Azure Monitor managed service for Prometheus to collect and analyze metrics at scale by using a Prometheus-compatible monitoring solution that's based on the Prometheus project from the Cloud Native Computing Foundation. You can use remote write to send data from Prometheus servers in your cluster to the Azure managed service.

Prerequisites

To send data from a Prometheus server by using remote write, you need:

Register an application with Microsoft Entra ID

  1. Complete the steps to register an application with Microsoft Entra ID and create a service principal.

  2. Copy the tenant ID and client ID of the service principal:

    1. In the Microsoft Entra admin center, go to Identity > Applications > App registrations, and then select your application.
    2. Go to the Overview page for the app.
    3. Copy and retain the Directory (tenant) ID value.
    4. Copy and retain the Application (client) ID value.

  3. Create a new client secret. Copy and retain the value of the secret.

  4. In your application code, set the values of the tenant ID, client ID, and client secret that you copied:

    export TENANT_ID=<tenant-id>
export CLIENT_ID=<client-id>
export CLIENT_SECRET=<client-secret>

Assign the Monitoring Metrics Publisher role to the application

The application must have the Monitoring Metrics Publisher role for the data collection rule that is associated with your Azure Monitor workspace.

  1. In the Azure portal, go to the instance of Azure Monitor for your subscription.

  2. On the resource menu, select Data Collection Rules.

  3. Select the data collection rule that is associated with your Azure Monitor workspace.

  4. On the Overview page for the data collection rule, select Access control (IAM).

  5. Select Add, and then select Add role assignment.

  6. Select the Monitoring Metrics Publisher role, and then select Next.

  7. Select User, group, or service principal, and then choose Select members. Select the application that you registered, and then choose Select.

  8. To complete the role assignment, select Review + assign.

Create a secret in your Azure Red Hat OpenShift cluster

To authenticate by using a remote write endpoint, you use the OAuth 2.0 authentication method from the supported remote write authentication settings.

To begin, create a secret by using the client ID and client secret:

cat << EOF | oc apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: oauth2-credentials
  namespace: openshift-monitoring
stringData:
  id: "${CLIENT_ID}"
  secret: "${CLIENT_SECRET}"
EOF

Set up remote write

To set up remote write for the default platform monitoring, update the cluster-monitoring-config config map YAML file in the openshift-monitoring namespace.

  1. Open the config map file for editing:

     oc edit -n openshift-monitoring cm cluster-monitoring-config

    data:
  config.yaml: |
    prometheusK8s:
      remoteWrite:
        - url: "<INGESTION-URL>"
          oauth2:
            clientId:
              secret:
                name: oauth2-credentials
                key: id
            clientSecret:
              name: oauth2-credentials
              key: secret
            tokenUrl: "https://login.microsoftonline.com/<TENANT_ID>/oauth2/v2.0/token"
            scopes:
              - "https://monitor.azure.com/.default"

  2. Update the config map file:

    1. Replace INGESTION-URL in the config map file with the value for Metrics ingestion endpoint from the Overview page for the Azure Monitor workspace.

    2. Replace TENANT_ID in the config map file with the tenant ID of the service principal.

Visualize metrics by using Azure Managed Grafana

You can use community Grafana dashboards to visualize the captured metrics, or you can create contextual dashboards.

  1. Create an Azure Managed Grafana workspace.

  2. Link the Azure Managed Grafana workspace to your Azure Monitor workspace.

  3. Import the community Grafana dashboard Openshift/K8 Cluster Overview (ID 3870) to the Grafana workspace.

  4. For the data source, use your Azure Monitor workspace.

  5. Save the dashboard.

To access the dashboard, in your Azure Managed Grafana workspace, go to Home > Dashboards, and then select the dashboard.

Troubleshoot

For troubleshooting information, see Azure Monitor managed service for Prometheus remote write.

Related content