Vinay Singh's blog https://docs.microsoft.com/archive/blogs/vinaysin/feed.xml Just another Developer Network site Wed, 17 Apr 2019 03:43:11 GMT en-US hourly 1 Calling Azure Functions from XSLT map in Logic Apps https://docs.microsoft.com/archive/blogs/vinaysin/calling-azure-functions-from-xslt-map-in-logic-apps Wed, 19 Jul 2017 18:55:50 GMT https://blogs.msdn.microsoft.com/vinaysin/?p=495 Scenario Let's say we have to build an expense processing application. It takes the receipt with amount and currency information and converts it into final amount in US dollars based on the exchange rate.

Solution

We will use Logic Apps to build the solution. It takes the receipt information through a Request trigger, uses XML Transform to generate the final amount and returns the result using Response action. We don't want to hard-code the currency conversion rate in the XSLT map as they keep changing and will Azure Functions to look it up. Let's start with the building blocks - Azure Function and XSLT map. Later we will stitch everything together in a Logic App.

Azure Function

We have a C# Webhook Function here which reads the currency information from request query parameters and gets the conversion rate to USD. Here we have hardcoded the mapping in the Function, but in a real world scenario it would be coming from a database or in real-time from a web service etc. Click here to download the source code.

XSLT Map

The XSLT map takes Azure Function App URL as a parameter (line #5). There are several benefits of doing this -
  1. The Function URL has authorization code embedded in it. We don't want to hard-code them in the map, rather fetch them from Azure Key Vault during deployment.
  2. We might have different Function App URL for test vs. production environment.
To learn more about XSLT parameters support in Logic Apps you can try out Azure Logic Apps - XSLT with parameters on github. It then creates the output XML consisting of original currency, original amount and calculated amount in USD. For the calculation it first call the Azure Function passing the currency as input parameter and then multiplies it with the original amount. We have used inline C# scripting to call the Function App using the HTTP client. In this example, we are passing the input parameters through query parameters and making a GET request but it can be easily modified to do more complex things like making a POST request with XML payload in request body. Click here to download the source code.

Logic App

Let's start with creating an Integration Account first and uploading the XSLT map into it. Next, create a Logic App in the same region and associate the Integration Account with it. Its a simple Logic App containing a Request trigger, XML Transform action and a Response action. The Request trigger exposes a REST endpoint and expects the receipt information (XML format) in the request body. The Transform action shows 2 fields -
  1. The Content field takes the receipt information from the trigger body.
  2. The Map field shows the list of maps in the associated Integration Account. Select the map uploaded in previous steps.
The designer detects that the map contains an input parameter called AzureFunctionUri and automatically adds another field in the action. Copy the Function URL from the Function blade and paste it here. In the end, we call the Logic App from Postman passing in the receipt payload in the request body and get back the amount in USD. ]]> Logic Apps Performance with Service Bus https://docs.microsoft.com/archive/blogs/vinaysin/logic-apps-performance-with-service-bus Fri, 14 Apr 2017 15:00:29 GMT https://blogs.msdn.microsoft.com/vinaysin/?p=415 Single message trigger Let’s create a new Logic App with the just the trigger “When a message is received in a queue (auto-complete)” provided by the Service Bus connector. single-message-trigger We pre-populated the queue with 10,000 messages. It takes 25.5 minutes for the Logic App to drain all messages from the queue, i.e. at the rate of 6.5 messages per second.

Batch messages trigger

Now let’s create another Logic App with the batch trigger “When one or more messages arrive in a queue (auto-complete)”. The trigger configuration are as follows –
  1. Maximum message count – starting from 5 going up to 175 (max limit) across test runs
  2. Polling frequency – 30 seconds
batch-messages-trigger Again, we pre-populated the queue with 10,000 messages. It takes just 6 minutes with batch size of 5 for the Logic App to drain all messages from the queue, i.e. at the rate of 27.5 messages per second. As we increase the batch size to the maximum allowed value of 175, the throughput goes up to 400 messages per second. batch-size-vs-throughput

Competing consumers

We can further increase the overall throughput with multiple Logic Apps (competing consumers scenario). Let’s create 10 Logic Apps with batch size of 175. We pre-populated the queue with 100K messages, i.e. 10 times more than previous runs to give enough time for competing logic apps to consume enough messages. It takes just 71 seconds for these Logic Apps to drain all messages from the queue, i.e. at the overall rate of 1408 messages per second. competing-consumers We should be able to scale further by adding more competing Logic Apps.

Summary

Here’s the result summary of various runs. results-summary]]> Get started with the Azure Event Hubs connector https://docs.microsoft.com/archive/blogs/vinaysin/get-started-with-the-azure-event-hubs-connector Sat, 25 Mar 2017 21:38:55 GMT https://blogs.msdn.microsoft.com/vinaysin/?p=325 Connect to Azure Event Hubs to send and receive events. You can perform operations such as send an event to an Event Hub and receive events from an Event Hub.

To use any connector, you first need to create a logic app. You can get started by creating a logic app now.

Connect to Event Bus

Before your logic app can access any service, you first need to create a connection to the service. A connection provides connectivity between a logic app and another service.

Prerequisites

You must have an Event Hubs account. Before you can use your Azure Event Hubs account in a logic app, you must authorize the logic app to connect to your Event Hubs account. Fortunately, you can do this easily from within your logic app on the Azure portal. Here are the steps to authorize your logic app to connect to your Event Hubs account:
  1. To create a connection to Event Hubs, in the logic app designer, select Show Microsoft managed APIs in the drop-down list. Then enter event hubs in the search box. Select the trigger or action you want to use.event_hubs_trigger_search
  2. If you haven't created any connections to Event Hubs before, you'll be prompted to provide your Event Hubs credentials. These credentials are used to authorize your logic app to connect to and access your Event Hubs’ data. The Event Hubs connector needs the connection string for the Event Hubs namespace. It also requires Manage permissions. A good way to know if your connection string is for the namespace or a specific entity is if it contains the EntityPath parameter. If it does, it is not the right connection string for a logic app.event_hubs_namespace
  3. After you have received the connection string for the namespace, you can use it for the API connection in Logic Apps.event_hubs_connection
  4. Notice the connection has been created, and you are now free to proceed with the other steps in your logic app.
  5. event_hubs_trigger

    Use an Event Hubs trigger

    A trigger is an event that can be used to start the workflow defined in a logic app. Learn more about triggers. Here's how to use the Event Hubs - When events are available in Event Hub trigger to initiate a logic app workflow when new events are sent to an Event Hub.
    Note
    You will be prompted to sign in with your Event Hubs connection string if you have not already created a connection to Event Hubs.
    1. In the search box on the logic apps designer, enter event hubs. Then select the Event Hubs - When events are available in an Event Hub event_hubs_trigger_search
    2. The When an event in available in an Event Hub dialog box is displayed.event_hubs_trigger_3
    3. Enter the name of the Event Hub you would like the trigger to monitor. Optionally, you can also select a consumer group.
    4. event_hubs_trigger_2
    At this point, your logic app has been configured with a trigger. When new events are available in the Event Hub you selected, the trigger will begin a run of the other triggers and actions in the workflow.

    Use an Event Hubs action

    An action is an operation carried out by the workflow defined in a logic app. Learn more about actions. Now that you have added a trigger, it's time to do something interesting with the data that's generated by the trigger. Follow these steps to add the Event Hubs - Send event action. This action sends an event to an Event Hub. Follow these steps to create the send event action:
    1. Select+ New step to add the action.
    2. Select Add an action. This opens a search box, where you can search for any action you would like to take. For this example, Event Hubs’ actions are of interest.nextaction
    3. Enter event hubs.
    4. Select Event Hubs - Send event as the action to takeevent_hubs_actions_search
    5. Enter the content for the event. This is required.
    6. Enter the event hub name to which the event will be sent. This is also required.
    7. Provide other details about the event. This is optional.send_event
    8. Save the changes to your workflow.savelogicapp
]]> Consuming Azure Stream Analytics output in Azure Logic Apps https://docs.microsoft.com/archive/blogs/vinaysin/consuming-azure-stream-analytics-output-in-azure-logic-apps Tue, 17 Jan 2017 13:33:59 GMT https://blogs.msdn.microsoft.com/vinaysin/?p=35 Stream Analytics Azure Stream Analytics lets users perform real-time analytics for their Internet of Things (IoT) solutions. In this article, we will use Event Hubs as a source and Service Bus Queue as a destination for Stream Analytics. Azure Event Hubs is a hyper-scale telemetry ingestion service that can store millions of events. And, Azure Service Bus is a highly-reliable cloud messaging service with Queues offering a First In, First Out (FIFO) message delivery to competing consumers.

Event Hubs (source)

Create an Event Hubs namespace in Azure Portal. create_eventhubs Then add a new Event Hub into the namespace. create_eventhub

Service Bus Queue (destination)

Create a Service Bus namespace in Azure Portal. create_sbnamespace Then add a new queue into the namespace. create_sbqueue

Stream Analytics job

Create a new Azure Stream Analytics job that processes messages coming through Event Hub and stores the output into Service Bus queue. For this post, we are using a pass-through Stream Analytics. create_streamanalyticsjob

Service Bus Queue Reader Logic App

The Service Bus Queue Reader Logic App reads a message from a Service Bus queue, uses Azure Functions to remove special characters that get added by Azure Stream Analytics and then sends it to a child Logic App for processing the message. Click here to download complete Logic App definition.

Service Bus Queue Trigger

Add a new Service Bus queue trigger to read messages from a Service Bus queue and trigger the Logic App. servicebus_receive_trigger

Special characters handling using Azure Functions

Azure Stream Analytics adds certain special (Unicode) characters in output messages which cause failures in Logic Apps when trying to decode them (e.g. to JSON format). As a result, we need to pre-process the message using Azure Functions and remove those special characters before going ahead with regular message processing in Logic Apps. Create a new C# Webhook Function App (let’s call it ParseStreamAnalyticsJSONContent) in Azure Portal and insert below code snippet into it. Create the Function App in same region as Logic Apps for being callable from Logic Apps. It is recommended to create it in same Resource Group as Logic Apps. Click here to download the source code. azurefunctions Then add a Function App action in Logic Apps and choose ParseStreamAnalyticsJSONContent function app that was created in previous step. parsejson_functionapp_action

Call Message Processor child Logic App

Call the Message Processor child Logic App that has logic for processing these messages, details of which are covered in next section. call_childworkflow_action

Message Processor Logic App

The Message Processor Logic App takes an event data in JSON format and send an e-mail using Office 365 Connector. Click here to download complete Logic App definition.

Request Trigger

Create a request trigger and provide it JSON schema for incoming message. JSON schema helps tokenize message properties which come handy in subsequent steps. request_trigger

Office365 Send Email

Send an email using Office 365 Connector. You can use message properties coming from the request trigger to craft the email template (subject, body etc.). send_office365_email

Response action

Add a response action to complete this Logic App. Having a Response action is a MUST to be able to call this Logic App from another Logic App. response_action]]> Hello world! https://docs.microsoft.com/archive/blogs/vinaysin/hello-world Sun, 28 Aug 2016 02:23:37 GMT https://blogs.msdn.microsoft.com/vinaysin/?p=1 Developer Network. This is your first post. Edit or delete it, then start blogging!]]>