Partner Center API and Azure Resource Utilization

Article
12/19/2016

Thanks to the hard work from the Partner Center team we are now able to query Azure usage records directly from the Partner Center API. With this update to the Partner Center API we can now perform everything that the CREST API can and a whole lot more! Numerous partners ask how they can generate an itemized invoices for their customer’s Azure consumption. Until now there was not a good way to accomplish this. With this post I want to focus on how to utilize Azure usage records to create an itemized invoice for Azure consumption. The remaining sections of this post will introduce the concepts and provide some sample code. All sample code provided in this post is provided as-is and with no warranty.

Azure Utilization Records

Azure utilization records provide information pertaining to the consumption of Azure services. When you supplement these records with details from the Azure RateCard it is possible to calculate the total charges that have been incurred. When we query the Partner Center API for usage records, using the utilization records feature, it returns JSON that is similar to the following

 
{
  "usageStartTime": "2016-12-12T16:00:00-08:00",
  "usageEndTime": "2016-12-13T16:00:00-08:00",
  "resource": {
    "id": "9cb0bde8-bc0d-468c-8423-a25fe06779d3",
    "name": "Standard IO - Table Write Operation Units (in 10,000s)",
    "category": "Data Management",
    "subcategory": ""
  },
  "quantity": 0.0146,
  "unit": "10,000s",
  "infoFields": {},
  "instanceData": {
    "resourceUri": "/subscriptions/8e4357c6-bd40-4a49-b169-ced26d105d70/resourceGroups/securitydata/providers/Microsoft.Storage/storageAccounts/myaccount",
    "location": "ussouthcentral",
    "partNumber": "",
    "orderNumber": ""
  },
  "attributes": { "objectType": "AzureUtilizationRecord" }
}

As you can use the record being returned contains a wealth of information. If you are using the Partner Center Managed API then it will serialize the response into an object that you can perform operations against. I would like to point out that the resource group name is not one of the properties included with a usage record. However, you can obtain the resource group name with a little bit of work by parse the resourceUri property. The following snippet of code shows one way to obtain all of the usage records for all of the subscription that the specified customer has

 
/// <summary>
/// Gets all Azure usage records for the specified customer. 
/// </summary>
/// <param name="customerId">Identifier for the customer of interest.</param>
/// <returns>A collection of Azure usage records.</returns>
/// <exception cref="ArgumentException">
/// customerId
/// </exception>
public Dictionary<string, List<AzureUtilizationRecord>> GetUsageRecords(string customerId)
{
    Dictionary<string, List<AzureUtilizationRecord>> usageRecords;
    ResourceCollection<Subscription> subscriptions;

    if (string.IsNullOrEmpty(customerId))
    {
        throw new ArgumentException(nameof(customerId));
    }

    try
    {
        subscriptions = PartnerCenter.Customers.ById(customerId).Subscriptions.Get();
        usageRecords = new Dictionary<string, List<AzureUtilizationRecord>>();

        foreach (Subscription s in subscriptions.Items.Where(x => x.BillingType == BillingType.Usage))
        {
            ResourceCollection<AzureUtilizationRecord> records =
                PartnerCenter.Customers.ById(customerId).Subscriptions.ById(s.Id).Utilization.Azure.Query(
                DateTime.Now.AddMonths(-1), DateTime.Now);

            usageRecords.Add(s.Id, new List<AzureUtilizationRecord>(records.Items));
        }

        return usageRecords;
    }
    finally
    {
        subscriptions = null;
    }
}

Now that we have a way to obtain Azure usage records the remaining steps in the process, of creating the appropriate invoice, are rather straight forward. With this post I am considering an itemized based invoice as one that groups resources based upon a tag with a key of costCenter. With all of this is in mind we can leverage code similar to the following to generate the invoice for a specific customer.

 
/// <summary>
/// Generates an itemized based invoice for the speicifed customer.
/// </summary>
/// <param name="customerId">Identifier assign to the customer of interest.</param>
/// <remarks>
/// This function will group resources based upon the value of the costCenter tag that 
/// is applied to the resource. All resources that do not have a costCenter tag will be 
/// grouped together and reported accordingly.
/// </remarks>
public void GenerateInvoice(string customerId)
{
    AzureRateCard rateCard;
    Dictionary<string, List<AzureUtilizationRecord>> usageRecords;
    List<AzureUtilizationRecord> costCenterRecords;
    List<AzureUtilizationRecord> records;
    ICollection<string> costCenters;

    if (string.IsNullOrEmpty(customerId))
    {
        throw new ArgumentException(nameof(customerId));
    }

    try
    {
        rateCard = PartnerCenter.RateCards.Azure.Get();
        usageRecords = GetUsageRecords(customerId);

        foreach (string subscription in usageRecords.Keys)
        {
            // Obtain the collection of usage records that either has no costCenter tag or no tags.
            records = usageRecords[subscription].Where(x => x.InstanceData.Tags == null ||
                x.InstanceData != null && !x.InstanceData.Tags.ContainsKey("costCenter")).ToList();

            // Process the usage records with no tags or no costCenter tag.
            ProcessUsageRecords(rateCard, subscription, string.Empty, records);

            // Obtain the collection of usage records that have a costCenter tag.
            records = usageRecords[subscription].Where(
                x => x.InstanceData.Tags != null && x.InstanceData.Tags.ContainsKey("costCenter")).ToList();

            // Obtain a list of the cost centers utilized.
            costCenters =
                records.Select(x => x.InstanceData.Tags)
                    .Where(x => x.Keys.Contains("costCenter"))
                    .Select(x => x.Values).SingleOrDefault();

            if (costCenters == null)
            {
                continue;
            }

            // Process the usage records with a costCenter tag. 
            foreach (string cs in costCenters)
            {
                costCenterRecords = records.Where(x =>
                    x.InstanceData.Tags.Keys.Contains("costCenter") &&
                    x.InstanceData.Tags["costCenter"].Equals(cs, StringComparison.CurrentCultureIgnoreCase)).ToList();

                ProcessUsageRecords(rateCard, subscription, cs, costCenterRecords);
            }
        }
    }
    finally
    {
        costCenterRecords = null;
        rateCard = null;
        records = null;
        usageRecords = null;
    }
}

private void ProcessUsageRecords(AzureRateCard rateCard, string subscription, string costCenter, IEnumerable<AzureUtilizationRecord> usageRecords)
{
    AzureMeter meter;
    decimal quantity = 0, rate = 0, total = 0;

    if (string.IsNullOrEmpty(subscription))
    {
        throw new ArgumentException(nameof(subscription));
    }

    if (usageRecords == null)
    {
        return;
    }

    try
    {
        Console.WriteLine(string.IsNullOrEmpty(costCenter)
            ? "Usage records with no cost center tag..."
            : $"Usage records for {costCenter} cost center...");

        foreach (AzureUtilizationRecord record in usageRecords)
        {
            // Obtain a reference to the meter associated with the usage record.
            meter = rateCard.Meters.Single(x => x.Id.Equals(record.Resource.Id));
            // Calculate the billable quantity by substracting the included quantity value.
            quantity = record.Quantity - meter.IncludedQuantity;

            if (quantity > 0)
            {
                // Obtain the rate for the given quantity. Some resources have tiered pricing
                // so this assignment statement will select the appropriate rate based upon the
                // quantity consumed, excluding the included quantity. 
                rate = meter.Rates
                    .Where(x => x.Key <= quantity).Select(x => x.Value).Last();
            }

            total += quantity * rate;

            Console.WriteLine("{0}, {1}", meter.Name, (quantity * rate));
        }

        Console.WriteLine($"Total Charges {total}");
    }
    finally
    {
        meter = null;
    }
}

Since all of the code in this post is considered sample code the invoice data is simply being written to the console. The following figure shows sample when you invoke the code

In a real world scenario I would recommend that you leverage something like an Azure WebJob to generate and store invoices in your particular billing system. Hopefully this information will help you the necessary invoices for your customers.

Comments

Anonymous
April 10, 2017
Great Post Isaiah,Do you know why ussage records had 8 hours difference from UTM?Thanks
- Anonymous
  April 18, 2017
  Hi Rodrigo, based on my understanding the end and start date for the usage is recorded in UTC. If you are seeing a discrepancy after you done the conversion of the timestamp to your local time and when the when the usage was actually utilized I would recommend that you reach out to our support organization. There is a chance you will see a potential 24 hour delay in usage records being available through the API. You can find out more about this at https://msdn.microsoft.com/en-us/library/azure/mt219001.aspx. Please let me know if you have any other questions.
Anonymous
September 05, 2017
Hi Isaiah, this looks exactly what I'm after but I can't see how you get to the tags, they aren't there in your sample JSON response.Also what schema/URL are you querying for this?
- Anonymous
  November 27, 2017
  Hi JimboWasEre, If any tags are present then they are returned in the instanceData field for the given resource. Otherwise, the Tag field is ommitted from the JSON that is returned when requesting utilization records. You can find additional information at https://msdn.microsoft.com/en-us/library/partnercenter/mt791774.aspx
Anonymous
November 20, 2017
Hi Isaiah, thanks for your great posts, can tell you really know your stuff! Forgive me if I'm being a bit dumb but your JSON sample doesn't show any key-value pair for tags that I can see? Where would they be? And can you show the exact API call to return the JSON that would contain it?
- Anonymous
  November 27, 2017
  Hi Jimbo, That is a good catch. When I was writting the post I extracted a single record and completely missed that it did not have any tags. When you are requesting utilization records, through the Partner Center API, if any tags are present then they will be returned as part of the instanceData element. The utilization record that I have in the post did not have any tags, so that field was omitted from what was returned. You can find additional information regarding the utilization request at https://msdn.microsoft.com/en-us/library/partnercenter/mt791774.aspx. I will work on getting this posted updated to include a more appropriate JSON example.
Anonymous
March 02, 2018
The comment has been removed
- Anonymous
  March 02, 2018
  Hi Rafael, Yes, that is correct. You can combine the information for Rate Card with the utilization record to obtain the price. I would recommend that you check out this webinar and this hands-on lab. Please let me know if you have any questions.

Share via

Partner Center API and Azure Resource Utilization

Tags

Azure Utilization Records

Comments

Additional resources