Azure Function App: Create a trigger on Cosmos DB
1 Introduction
This article is going to describe how to create a trigger/continuous checking on a data set using an Azure function app, this article has used Azure CosmosDB as a data set with multiple documents. Sample application shows how to get an alert or a notification when a vehicle is going to violate the speed limit in a road or in a highway. Let's dig into deep and see how to use Azure function app to find a solution to this scenario
2 Background
If you are interested in Azure and its features, check these articles
- Check this article if you want to know more about Azure Cosmos DB and its features, https://social.technet.microsoft.com/wiki/contents/articles/38326.explore-azure-cosmos-db-document-api.aspx
- If you want to know how to design a complex database structure with relationships and how to query in Azure Cosmos DB, go through this article, https://social.technet.microsoft.com/wiki/contents/articles/39421.azure-cosmos-db-designing-your-data-structure.aspx
- When you want to use Azure blob storage to store some of your documents, image files, read this article, https://social.technet.microsoft.com/wiki/contents/articles/37981.send-smtp-email-with-azure-blob-storage-attachments.aspx
3 Create a document in Azure Cosmos DB
3.1 Create Azure Cosmos DB account
Go to Azure Cosmos DB and navigate to Azure portal, Click on Add button, It will show Azure Cosmos DB account creation page.
Add account name and select an *API *from list. In this example, we want to work with a simple document collection, so choose SQL as the *data API
*
You can select a subscription for billing of the account as below.
You can create a new resource group or use an existing one.
Select account location as a nearest region to your clients.
You can see a option to enable geo redundancy. At this point we are not going to replicate this data into another region, so don't tick that option. You can later replicate your data into different places to achieve high availability of your data.
You will be redirected to Quick Start page, you can create a collection in various platforms, in this case let's use .NET Core.
You can get an idea what is the default storage of your collection, what will be the throughput and estimated hourly billing of the collection.
3.2 Create a document in a collection
Let's try to create a document in Azure Cosmos DB collection,
Go to Data Explorer, your new database account is created.
Create a new collection, We can create a document in that collection. Lets add some test data and create a document as below.
{
"id": "replace_with_new_document_id",
"firtsName": "Hansamali",
"lastName": "Gamage"
}
3.3 Query a collection
Click on your collection and click on New SQL Query, In the New Query window, write the simplest query and execute it.
You can see inserted document as a result, and it has required *2.28 request units *to run the query
4 Create a document programmatically
We created a document in Azure Cosmos DB collection using Azure portal, Let's see how we can create a document in Azure Cosmos DB programatically
4.1 Create a console application
Open Visual Studio, This example has used Visual Studio 2017 with .NET Core 2.0.3 version to create the application. Select .NET Core and Console App (.NET Core), You can give a name and click on OK button
We need to add necessary references to access Cosmom DB,
Go to Tools -> NuGet Package Manager -> Manage NuGet Packages for solution, Search for Azure Document DB, it will show as below.
This version of Cosmos DB references are not compatible with .NET Core 2.0. So we have to find another one
Search for Microsoft.Azure.DocumentDB.Core, then you can find a reference that compatible with .NET Core. This is a special version of Document DB library designed for .NET Core application
4.2 Create Cosmos DB settings
Create a class named DBConnector and inside that, create a method as follows
First we need to create a DocumentClient to connect to the database account, we have to pass endpoint url and auth key
static class DBConnector
{
static DocumentClient client;
public async static Task Create ()
{
try
{
string endPointUrl = "";
string primaryKey = "";
}
}
}
4.3 Create a Document Client
We have to instantiate a DocumentClient object to connect to Azure Cosmos DB account, in order to do that, we have to pass service endpoint URL and auth key or a token, Let's see how we can find them
Go to Keys section in Azure Cosmos DB account, Get the URI and Primary Key to connect to Azure Cosmos DB account
static DocumentClient client;
public async static Task Create ()
{
try
{
string endPointUrl = "https://highwaytraffic.documents.azure.com:443/";
string primaryKey = "jSuZYTAvpOiw65P1O1TEPD7pAmjTTDUd5I9pudCkABUz97f3fxcnjg259o7jqDNCJXDyRAQrjOm6H2k24ZjyrQ==";
}
Create a DocumentClient by passing endpoint URL and key
client = new DocumentClient(new Uri(endPointUrl), primaryKey);
4.4 Create the database
Create the database by passing database name
public async static Task CreateDatabase (string database)
{
var db = await client.CreateDatabaseIfNotExistsAsync(new Database { Id = database});
}
4.5 Create a Collection
Create a document collection by passing database name and collection name
public async static Task CreateCollection(string database, string collection)
{
var databasecollection = await client.CreateDocumentCollectionIfNotExistsAsync(UriFactory.CreateDatabaseUri(database),
new DocumentCollection { Id = collection });
}
In the main method, pass database and collection name and call necessary methods
string databaseName = "trafficdata";
string databaseCollection = "vehiclespeed";
client = new DocumentClient(new Uri(endPointUrl), primaryKey);
await CreateDatabase(databaseName);
await CreateCollection(databaseName, databaseCollection);
call database create method from main program as below,
namespace highwaytraffic
{
class Program
{
static void Main(string[] args)
{
DBConnector.Create().Wait();
}
}
}
Let's run the application and see database is created in Azure account, Go to Overview tab in Cosmos DB account and check whether database is created. In Collections tab, it shows database and *collectionid
*
4.6 Create a document in a collection
In order to create a document, we should have a object,
In this example we try to find vehicles who exceeds certain speed limit, so need to create a class to store vehicle speed,
Add a class called VehicleSpeed into the solution,
namespace highwaytraffic
{
public class VehicleSpeed
{
[JsonProperty(PropertyName = "vehicleNumber")]
public string VehicleNumber { get; set; }
[JsonProperty(PropertyName = "speed")]
public double Speed { get; set; }
[JsonProperty(PropertyName = "city")]
public string City { get; set; }
}
}
Create a document by passing database and collection ids, And specially you should pass an instance from a VehicleSpeed class
public async static Task CreateDocument (string database, string collection, VehicleSpeed vehicleSpeed)
{
await client.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri(database, collection), vehicleSpeed);
}
Create an instance from VehicleSpeed class, and pass it to the document create method
VehicleSpeed vehicleSpeed = new VehicleSpeed { VehicleNumber = "KJ -7788",
City = "Colombo", Speed = 100.78 };
await CreateDocument(databaseName, databaseCollection, vehicleSpeed);
Let's run this latest code and see whether document is created,
Click on Data Explorer, then it shows document inside the collection, Click on Documents, you can see created document with few other *meta data
*
4.7 Create multiple documents in a collection
We have to create multiple documents to demo this application, let's see how we can do that,
Create a for loop and insert few documents with a random speed value as below
public async static Task CreateMultipleDocument(string database, string collection)
{
int documentCount = 500;
for (int i = 0; i < documentCount; i++)
{
double speed = new Random().NextDouble() * 100;
VehicleSpeed vehicleSpeed = new VehicleSpeed { VehicleNumber = "KJ -7788",
City = "Colombo", Speed = speed};
await client.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri(database,
collection), vehicleSpeed);
}
}
Let's call this method and check whether it works,
await CreateMultipleDocument(databaseName, databaseCollection);
Let's run the application and see how it works, Go to Documents, it shows inserted documents as follows,
Click on New SQL Query and Get a count of documents as follows,
SELECT COUNT(1) FROM c
We created required no of documents for this demo, You can find the full code from below,
static DocumentClient client;
public async static Task Create ()
{
try
{
string endPointUrl = "https://highwaytraffic.documents.azure.com:443/";
string primaryKey = "jSuZYTAvpOiw65P1O1TEPD7pAmjTTDUd5I9pudCkABUz97f3fxcnjg259o7jqDNCJXDyRAQrjOm6H2k24ZjyrQ==";
string databaseName = "trafficdata";
string databaseCollection = "vehiclespeed";
client = new DocumentClient(new Uri(endPointUrl), primaryKey);
await CreateDatabase(databaseName);
await CreateCollection(databaseName, databaseCollection);
VehicleSpeed vehicleSpeed = new VehicleSpeed { VehicleNumber = "KJ -7788",
City = "Colombo", Speed = 100.78 };
await CreateDocument(databaseName, databaseCollection, vehicleSpeed);
await CreateMultipleDocument(databaseName, databaseCollection);
}
catch (DocumentClientException de)
{
Exception baseException = de.GetBaseException();
Console.WriteLine("{0} error occurred: {1}, Message: {2}", de.StatusCode, de.Message, baseException.Message);
}
catch (Exception e)
{
Exception baseException = e.GetBaseException();
Console.WriteLine("Error: {0}, Message: {1}", e.Message, baseException.Message);
}
}
5 Create Azure Function App
5.1 Create a basic Function App
Let's see how we can create a azure function app in the portal, Click on New icon,
In next screen, click on Compute and it shows Function App, click on it
You can see the following create screen,
Add application name, subscription, resource group and select an operation system,
You have to select App Service plan as the Hosting plan, Hosting plan is going to define how resources are going to allocate for your function app,
If we select Consumption plan, resources are dynamically added to your application as per requirements, and you only pay for the time your function app runs,
If you select App service plan, your function app runs on a dedicated VM similar to Web apps, API apps and mobile apps, your function app is always running, Let's select App service plan as the hosting plan.
You have to create a storage account for your function app to use as below
We have defined all the configurations for function app to start, Let's go into the next step,
You will be prompted to this screen when function app created,
You can find App URL, status of the function app from this screen
5.2 Create a new function
We have to create a new function inside this Function app, Click on + icon as below,
You will see this screen, Click on Custom Function to view available templates to create a *Function App
*
You can see available templates as below,
5.3 Create CosmosDB trigger
We are going to create a trigger on Azure Cosmos DB and call a function based on that, So select CosmosDBTrigger - C# template, Then you can see what are the options you want to provide,
In Azure Cosmos DB trigger section, You can see Azure Cosmos DB account connection, we have to provide a Cosmos DB account to proceed with the trigger, click on new link
You can see available database accounts as below, Let's select our existing *Azure Cosmos DB
*
Give a name to your function, You can see new Cosmos DB account connection has been created as below,
Click on show value in front of Azure Cosmos DB account connection, You can see endpoint URL to Cosmos DB account and the auth key as below
Provide Database name & Collection name to the data set we are going to work on,
You can see *Collection name for leases *textbox,
Azure Cosmos DB Triger use Cosmos DB Change feed to listen to the changes, This trigger needs a secondary collection to store leases over the partitions,
Database collection and leases collection both should be available for the trigger to work
Go to *trafficdata database *and expand the collections, You can see leases collection has been created automatically,
You can see available *documents *as follows, Those documents contains *metadata *about data partitions
5.4 Run CosmosDB trigger
If you go into the Function App, you can see new function is created, Let's Run this application and see what happens,
You can see Logs windows appears as below, it shows Function start and completion, And it logs no new traces available in each minute,
Go to Visual studio and modify *Create() *method to insert a new *document *into monitored collection, Let's see this function app can trigger that
You can see logs window, Since collection got changed, Function has been started, and it has gone inside the if condition and printed document id as below,
6 Change basic Function
Let's change function app logic to detect high speed of the vehicle as follows,
Add a for loop in Run method, its going to detect a high speed value and print it in the screen
If you want, you can copy high speed vehicles into another collection, may be can store it in a blob
public static void Run(IReadOnlyList<Document> input, TraceWriter log)
{
if (input != null && input.Count > 0)
{
log.Verbose("Documents modified " + input.Count);
log.Verbose("First document Id " + input[0].Id);
for(int i = 0; i < input.Count; i++)
{
double speed = input[i].GetPropertyValue<double>("speed");
if(speed > 100)
{
string vehicleNo = input[i].GetPropertyValue<string>("vehicleNumber");
string city = input[i].GetPropertyValue<string>("city");
log.Verbose(string.Format("High speed detected in {0}, Vehicle No {1} and Speed {2}"
, city, vehicleNo, speed));
}
}
}
}
Change the speed of the vehicle as 112.78, and run the solution
New document is created in the Cosmos DB collection and you can see the log window as below
It shows 'High speed detection' message.
We have created a trigger on Azure Cosmos DB and it fires when a change occur in a document,
*
*Cosmos DB has a new feature called Cosmos DB change feed, it can continuously read batch of changed entities, it can track list of changed documents, not only documents, graphs, mongo db documents as well
When you query, as a result you get list of changed documents, list is going to depend on the throughput of the collection
Let's say you got a fatal error in the middle of the process, you can restart and continue from the recent change token, it's not going to miss any change
7 Download
7.1 Tech Net Gallery
You can download sample application from this location, https://gallery.technet.microsoft.com/Create-Cosmos-DB-Documents-e4d8ae00
7.2 GitHub
You can clone the source code from here, https://github.com/hansamaligamage/cosmoscollection-create
8 Conclusion
This article explains how to create a Azure Cosmos DB database, collection & documents using Azure Cosmos DB SDK. We can track changes on a collection by using Azure Cosmos DB change feed. Application explains how to run a trigger on Azure Cosmos DB data set and perform an action using a Azure Function app.
9 See Also
- Azure Cosmos DB: Designing your data structure
- Explore Azure Cosmos DB - document API
- Send smtp email with Azure blob storage attachments