Tutorial: Export carbon optimization data using a Python script

This tutorial explains how you use a Python script to export Azure Carbon optimization emissions data as JSON output. The Python script runs Carbon Service REST API network requests to get emissions data for your Azure resources. The script simplifies the process of querying the API and handling the response. You run the script to generate the JSON files and then use them for detailed analysis and reporting.

This tutorial covers the steps needed to export carbon optimization emissions data on your local Windows computer.

There are several steps to complete this tutorial:

• Review prerequisites and install missing components
• Download and install Python
• Install required Python packages
• Examine and update the Python script example
• Run the Python script
• Review the JSON output files

Prerequisites

Before you use the Python script, ensure you have:

• Appropriate Azure permissions (Carbon Optimization Reader, Subscription Owner, or Subscription Contributor role)
  • For more information, see Assign Azure roles by using the Azure portal and Assign access to Carbon optimization in Azure.
• A list of valid Azure subscription IDs that you want to get emissions data for
• An understanding of carbon emission scopes (Scope1, Scope2, Scope3)
• At least one full month of emissions data available for export (data is available for the previous month by day 19 of the current month)

Install Azure PowerShell

If you don't have Azure PowerShell installed on your local Windows computer, follow these steps:

Launch Windows PowerShell 5.1 as an administrator and run the following command to update PowerShellGet by using the PowerShell gallery:

Install-Module -Name PowerShellGet -Force

Set the PowerShell execution policy to remote signed or less restrictive:

• Check the PowerShell execution policy:

  Get-ExecutionPolicy -List
  

• Set the PowerShell execution policy to remote signed:

  Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
  
  

Use the Install-Module cmdlet to install the Az PowerShell module:

Install-Module -Name Az -Repository PSGallery -Force

You might need to restart PowerShell after the installation is complete.

For more information about installing Azure PowerShell, including installation on different platforms, see How to install Azure PowerShell.

Sign in to Azure

To start managing your Azure resources with the Az PowerShell module, launch a PowerShell session and run Connect-AzAccount to sign in to Azure:

Connect-AzAccount

Use your Azure account sign in credentials to log into the browser window that opens.

You need to repeat this step for every new PowerShell session you start.

Install Azure CLI

If you don't have Azure CLI installed on your local Windows computer, follow these steps:

1. Review the information at Install the Azure CLI on Windows.
2. Choose an installation method and then run the installer that's best suited for your local computer.
3. After installation completes, open a new command prompt or PowerShell window (with administrator privileges) sign in to your Azure account:

   az login
   

Download and install Python

If you don't have Python installed on your local Windows computer, follow these steps:

1. Review the information at Python downloads.
2. Choose, download, and run the installer for a Python release that's best suited for your local computer.

Install required Python packages

In your command prompt or PowerShell window (with administrator privileges), run the following command to install the required Python packages:

pip install azure-identity azure-mgmt-carbonoptimization

Examine and update the Python script example

1. Copy the following Python example script and save it locally. For example, save it as export_carbon_emission_data.py
2. Review the script and replace all instances of Subscription_ID_XXX with your actual Azure subscription IDs. Add or remove example subscriptions, as necessary. Up to 100 subscription IDs are supported.
3. Save the file.

By default, the script only outputs data for the last full month. You can update the script for a custom time range (full months) for some reports. Review the script comments to see if a report supports a custom range. Look for instances of:

    date_range = DateRange(
        start=date.fromisoformat(available_date_range.start_date),
        end=date.fromisoformat(available_date_range.end_date),
    )

Then update like the following example:

    date_range = DateRange(
        start=date.fromisoformat("YYYY-MM-DD"),
        end=date.fromisoformat("YYYY-MM-DD"),
    )

"""
Azure Carbon Optimization

This script queries Azure Carbon Optimization reports by using the Python azure-mgmt-carbonoptimization(https://pypi.org/project/azure-mgmt-carbonoptimization/) SDK.

"""

import os
import json
import logging
import time
from datetime import datetime, timezone, date
from typing import List, Dict, Any, Optional
from pathlib import Path

from azure.identity import DefaultAzureCredential
from azure.mgmt.carbonoptimization import CarbonOptimizationMgmtClient
from azure.mgmt.carbonoptimization.models import CarbonEmissionDataAvailableDateRange, CategoryTypeEnum, SortDirectionEnum,\
    OrderByColumnEnum, ReportTypeEnum, ResponseDataTypeEnum, CarbonEmissionItemDetailData, DateRange, EmissionScopeEnum,\
    CarbonEmissionData, ResourceCarbonEmissionItemDetailData, ResourceGroupCarbonEmissionItemDetailData, \
    ItemDetailsQueryFilter, MonthlySummaryReportQueryFilter, TopItemsSummaryReportQueryFilter, \
    TopItemsMonthlySummaryReportQueryFilter, OverallSummaryReportQueryFilter, CarbonEmissionDataListResult

from azure.core.exceptions import (
    ClientAuthenticationError, 
    HttpResponseError, 
    ResourceNotFoundError
)


# Initialize Azure CarbonOptimization SDK clients
credential = DefaultAzureCredential()
client = CarbonOptimizationMgmtClient(credential=credential)
carbon_service = client.carbon_service


def query_item_detail_data_by_category_type(category_type: CategoryTypeEnum) -> None :
    # get latest available carbon data date range
    available_date_range = carbon_service.query_carbon_emission_data_available_date_range()

    # Get latest month

    date_range = DateRange(
        start=date.fromisoformat(available_date_range.end_date),
        end=date.fromisoformat(available_date_range.end_date),
    )

    # Build query filter for item detail report
    item_detail_query_filter = ItemDetailsQueryFilter(date_range=date_range,
                                                      subscription_list=[
                                                          "Subscription_ID_001", "Subscription_ID_002", "Subscription_ID_100"
                                                      ], # suggest to put 100 subscription id 
                                                      carbon_scope_list=[EmissionScopeEnum.SCOPE1, EmissionScopeEnum.SCOPE2, EmissionScopeEnum.SCOPE3],
                                                      category_type=category_type,
                                                      order_by=OrderByColumnEnum.ITEM_NAME,
                                                      sort_direction=SortDirectionEnum.DESC,
                                                      page_size=50  # suggest with 2000 as pageSize
                                                      )

    with open(f"carbon_emission_{str(category_type.value).lower()}_item_detail_report.json", "a", encoding="utf-8") as f:
        while True:
            result_list = carbon_service.query_carbon_emission_reports(item_detail_query_filter)

            for item in result_list.value:
                f.write(json.dumps(item.as_dict(), ensure_ascii=False))
                f.write("\n")

            if not result_list.skip_token:
            # no more pages, break
                print("all data retrieved")
                break
        
            # set the continuation token for the next page
            item_detail_query_filter.skip_token = result_list.skip_token
            print("continue to get next page data")


def query_top_items_monthly_report_by_category_type(category_type: CategoryTypeEnum) -> None :
    # get latest available carbon data date range
    available_date_range = carbon_service.query_carbon_emission_data_available_date_range()

    date_range = DateRange(
        start=date.fromisoformat(available_date_range.start_date),
        end=date.fromisoformat(available_date_range.end_date),
    )

    # Build query filter for top items monthly report
    item_detail_query_filter = TopItemsMonthlySummaryReportQueryFilter(
        date_range=date_range,
        subscription_list=[
            "Subscription_ID_001", "Subscription_ID_002", "Subscription_ID_100"
        ], # suggest to put 100 subscription id 
        carbon_scope_list=[EmissionScopeEnum.SCOPE1, EmissionScopeEnum.SCOPE2, EmissionScopeEnum.SCOPE3],
        category_type=category_type,
        top_items=5
    )

    with open(f"carbon_emission_{str(category_type.value).lower()}_top_items_monthly_report.json", "a", encoding="utf-8") as f:
        result_list = carbon_service.query_carbon_emission_reports(item_detail_query_filter)

        for item in result_list.value:
            f.write(json.dumps(item.as_dict(), ensure_ascii=False))
            f.write("\n")

def query_top_items_report_by_category_type(category_type: CategoryTypeEnum) -> None :
    # get latest available carbon data date range
    available_date_range = carbon_service.query_carbon_emission_data_available_date_range()

    # only set one month for Top Items Report
    date_range = DateRange(
        start=date.fromisoformat(available_date_range.end_date),
        end=date.fromisoformat(available_date_range.end_date),
    )

    # Build query filter for top items report
    item_detail_query_filter = TopItemsSummaryReportQueryFilter(
        date_range=date_range,
        subscription_list=[
            "Subscription_ID_001", "Subscription_ID_002", "Subscription_ID_100"
        ], # suggest to put 100 subscription id 
        carbon_scope_list=[EmissionScopeEnum.SCOPE1, EmissionScopeEnum.SCOPE2, EmissionScopeEnum.SCOPE3],
        category_type=category_type,
        top_items=5
    )

    with open(f"carbon_emission_{str(category_type.value).lower()}_top_items_report.json", "a", encoding="utf-8") as f:
        result_list = carbon_service.query_carbon_emission_reports(item_detail_query_filter)

        for item in result_list.value:
            f.write(json.dumps(item.as_dict(), ensure_ascii=False))
            f.write("\n")

def query_overall_summary_report_by_category_type() -> None :
    # get latest available carbon data date range
    available_date_range = carbon_service.query_carbon_emission_data_available_date_range()

    date_range = DateRange(
        start=date.fromisoformat(available_date_range.start_date),
        end=date.fromisoformat(available_date_range.end_date),
    )

    # Build query filter for overall summary report
    item_detail_query_filter = OverallSummaryReportQueryFilter(
        date_range=date_range,
        subscription_list=[
            "Subscription_ID_001", "Subscription_ID_002", "Subscription_ID_100"
        ], # suggest to put 100 subscription id 
        carbon_scope_list=[EmissionScopeEnum.SCOPE1, EmissionScopeEnum.SCOPE2, EmissionScopeEnum.SCOPE3]
    )

    with open(f"carbon_emission_overall_summary_report.json", "a", encoding="utf-8") as f:
        result_list = carbon_service.query_carbon_emission_reports(item_detail_query_filter)

        for item in result_list.value:
            f.write(json.dumps(item.as_dict(), ensure_ascii=False))
            f.write("\n")

def query_monthly_overall_summary_report_by_category_type() -> None :
    # get latest available carbon data date range
    available_date_range = carbon_service.query_carbon_emission_data_available_date_range()

    date_range = DateRange(
        start=date.fromisoformat(available_date_range.start_date),
        end=date.fromisoformat(available_date_range.end_date),
    )

    # Build query filter for monthly overall summary report
    item_detail_query_filter = MonthlySummaryReportQueryFilter(
        date_range=date_range,
        subscription_list=[
            "Subscription_ID_001", "Subscription_ID_002", "Subscription_ID_100"
        ], # suggest to put 100 subscription id 
        carbon_scope_list=[EmissionScopeEnum.SCOPE1, EmissionScopeEnum.SCOPE2, EmissionScopeEnum.SCOPE3]
    )

    with open(f"carbon_emission_monthly_overall_summary_report.json", "a", encoding="utf-8") as f:
        result_list = carbon_service.query_carbon_emission_reports(item_detail_query_filter)

        for item in result_list.value:
            f.write(json.dumps(item.as_dict(), ensure_ascii=False))
            f.write("\n")

def get_latest_available_carbon_data_date_range() -> CarbonEmissionDataAvailableDateRange:
    """
    Query the latest available carbon data date range.
    """
    available_date_range = carbon_service.query_carbon_emission_data_available_date_range()

    print(f"Available date range: {available_date_range.start_date} to {available_date_range.end_date}")

    return available_date_range

if __name__ == "__main__":
    # get latest available carbon data date range
    get_latest_available_carbon_data_date_range()

    # get carbon emission item detail report
    query_item_detail_data_by_category_type(CategoryTypeEnum.RESOURCE)
    query_item_detail_data_by_category_type(CategoryTypeEnum.LOCATION)
    query_item_detail_data_by_category_type(CategoryTypeEnum.RESOURCE_TYPE)
    query_item_detail_data_by_category_type(CategoryTypeEnum.RESOURCE_GROUP)
    query_item_detail_data_by_category_type(CategoryTypeEnum.SUBSCRIPTION)

    # get top items monthly report
    query_top_items_monthly_report_by_category_type(CategoryTypeEnum.RESOURCE)
    query_top_items_monthly_report_by_category_type(CategoryTypeEnum.RESOURCE_GROUP)
    query_top_items_monthly_report_by_category_type(CategoryTypeEnum.LOCATION)
    query_top_items_monthly_report_by_category_type(CategoryTypeEnum.RESOURCE_TYPE)
    query_top_items_monthly_report_by_category_type(CategoryTypeEnum.SUBSCRIPTION)

    # get top items report
    query_top_items_report_by_category_type(CategoryTypeEnum.RESOURCE)
    query_top_items_report_by_category_type(CategoryTypeEnum.RESOURCE_GROUP)
    query_top_items_report_by_category_type(CategoryTypeEnum.LOCATION)
    query_top_items_report_by_category_type(CategoryTypeEnum.RESOURCE_TYPE)
    query_top_items_report_by_category_type(CategoryTypeEnum.SUBSCRIPTION)

    # get overall summary report
    query_overall_summary_report_by_category_type()

    # get monthly overall summary report
    query_monthly_overall_summary_report_by_category_type()

Run the Python script

Run the Python script from your command prompt or PowerShell window (with administrator privileges):

python export_carbon_emission_data.py

JSON output files are created in the same directory as the script. The files are named by report type.

Here's a list of the output files that are created:

• carbon_emission_location_item_detail_report.json
• carbon_emission_location_top_items_monthly_report.json
• carbon_emission_location_top_items_report.json
• carbon_emission_monthly_overall_summary_report.json
• carbon_emission_overall_summary_report.json
• carbon_emission_resourcegroup_item_detail_report.json
• carbon_emission_resourcegroup_top_items_monthly_report.json
• carbon_emission_resourcegroup_top_items_report.json
• carbon_emission_resourcetype_item_detail_report.json
• carbon_emission_resourcetype_top_items_monthly_report.json
• carbon_emission_resourcetype_top_items_report.json
• carbon_emission_resource_item_detail_report.json
• carbon_emission_resource_top_items_monthly_report.json
• carbon_emission_resource_top_items_report.json
• carbon_emission_subscription_item_detail_report.json
• carbon_emission_subscription_top_items_monthly_report.json
• carbon_emission_subscription_top_items_report.json

Review the JSON output files

Here's example output for the monthly overall summary report, from the carbon_emission_overall_summary_report.json file.

{"dataType": "OverallSummaryData", "latestMonthEmissions": 13871.2808902499, "previousMonthEmissions": 14007.1957894844, "monthOverMonthEmissionsChangeRatio": -0.00970321977912344, "monthlyEmissionsChangeValue": -135.91489923458}

For more information about the reports and the data they contain, see Export emissions API reference.

Next step

Export emissions API reference