1. Microsoft Flow: ideally for Business Users but powerful enough to tackle simple orchestration tasks
2. Logic Apps: Azure's purpose-built workflow engine with robust retry & failure capabilities, running on the same execution layer as your Azure deployments
3. Azure Functions + Logic Apps: for when your tasks are more complex than a simple Logic App can handle

And one pretty involved approach - using Azure Functions in conjunction with state storage and messaging for resilient execution of a chained or parallel workflow across many Azure Functions.

Now let's have a look at the newest kid on the block: Azure Functions' Durable Functions extension. Just announced at build but in preview for a few months prior, it's a code-first approach to workflow and orchestration using Azure Functions.

Durable Functions is only GA on Azure Functions v1. Therefore many of the constructs it uses are from the v1 namespaces (eg: HttpRequestMessage, HttpResponseMessage) however you are free to use these in a v2 Function without issue.

Durable Functions sits on top of Azure Functions and is available simply as a nuget package. What you get with this is a set of Function Trigger bindings and attributes that serve to abstract away the state management and queuing you saw in the Functions + Messaging approach from Part 3. Let's dive in.

Step 1: Augment our current Azure Function in to a Durable Function

Every Durable Function workflow has a Starter, one or more Orchestrators, and one or more Activity functions. As we saw in the last installment, our /Validate function serves as our starter here, so let's change it in to a Durable Functions Orchestration Client (aka Starter):

1. Add an [OrchestrationClient] parameter

public static IActionResult Run([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]Person req,
    [OrchestrationClient]DurableOrchestrationClient client,
    [ServiceBus(@"function-a", Connection = @"ServiceBusOrchestrationConnection")]out QueueMessage functionAmsg,
    TraceWriter log)

2. Add a call to kick off the new orchestration:

var orchestrationInstanceId = await client.StartNewAsync(@"Start", req);

3. Return either an HTTP 202 ACCEPTED "check for status" response:

return client.CreateCheckStatusResponse(req, orchestrationInstanceId);

At this point you'll need to convert this starter function to an async method which will leave you with a couple of issues:

Error	CS1988	Async methods cannot have ref or out parameters	Functions

This is because of that leftover out parameter for the Service Bus queue message. With DF we don't need this (it handles the queuing for you) so you're free to remove this parameter.

The other thing you'll notice is the call to CreateCheckStatusResponse takes an HttpRequestMessage as its first parameter which we don't have. You need to forego the "auto-deserialize" of the request body in to a Person object now, and field the actual request, then manually deserialize it in to a Person to send on to the orchestration function. At the end of the day, our new /Validate should look like this:

[FunctionName("Validate")]
public static async Task<HttpResponseMessage> RunAsync([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]HttpRequestMessage req,
    [OrchestrationClient]DurableOrchestrationClient client,
    TraceWriter log)
{
    var body = await req.Content.ReadAsStringAsync();
    var personObj = Newtonsoft.Json.JsonConvert.DeserializeObject<Person>(body);
    log.Info($@"Person request received: 
{body}");
    var orchestrationInstanceId = await client.StartNewAsync(@"Start", personObj);

    return client.CreateCheckStatusResponse(req, orchestrationInstanceId);
}

Step 2: Write the new Orchestrator function

As you saw above, our Starter executes a StartNewAsync with a string parameter for an orchestrator Function name. Well, behind the scenes DF looks for a Function with a matching name and an OrchestrationTrigger attribute. So we need to create one of these. This function will then be orchestrating the actual work of executing all the necessary activities.

Here's what just such a function looks like when it's defined

[FunctionName(nameof(Start))]
public static void Start([OrchestrationTrigger]DurableOrchestrationContext context)
{

}

There's a pretty important catch when writing Durable Functions and here's where it comes in to play. You might think that you could write all your calls to other functions within this orchestrator as

_client.PostAsync(...

calls but you'd be wrong. DF will blow up at runtime with a TPL error complaining you're trying to do an asynchronous operation. Any async method call not present on the Durable Functions API surface must be done in DF Activity functions and nowhere else (you can read more on this and other code constraints in the docs here).

Step 3: Write the worker "Activity Function"

So let's transform CheckFirstName in to an Activity function:

The first step is to change the trigger type to be an ActivityTrigger which gets a DurableActivityContext instance. You'll also notice that this activity function can return a value right back to the caller; not having to store it elsewhere; much more idiomatic, don't you think?

[FunctionName("CheckFirstName")]
public static Error CheckFirstName([ActivityTrigger]DurableActivityContext context, TraceWriter log)
{

Next, get the input passed to our activity (a Person object if you recall) via the context's GetInput<T> function call:

var person = context.GetInput<Person>();

As an added convenience you can also use the strong type of the input value instead of a context object (eg: Person instead of DurableActivityContext). Downside is you can't get the InstanceId from that, so I stuck with DurableActivityContext here.

And of course now do the processing & return a new Error object (or null). The whole activity function ends up looking like this:

[FunctionName("CheckFirstName")]
public static Error CheckFirstName([ActivityTrigger]DurableActivityContext context, TraceWriter log)
{
    var person = context.GetInput<Person>();
    log.Info($@"Message received: {context.InstanceId}");
    if ((person?.Name?.First?.Length > 1) == false)
    {
        var err = new Error { id = 1, message = "First name is null or not longer than 1 character" };
        log.Info($@" - Error found: {err.message}");
        return err;
    }
    else
    {
        log.Info($@" - No error found");
        return null;
    }
}

Notice how, where before we had defined a session id on our own and decorated our incoming request with it, Durable Functions has the instance id for the orchestration as part of the context it passes around; so we can use this in any log messages to correlate activity together.

Step 4: Tie it all together

Now that we have a worker, our orchestrator needs to call it. Back in Start, fill it out with a call to our activity function passing the request Person object we got in:

[FunctionName(nameof(Start))]
public static async Task<IList<Error>> Start([OrchestrationTrigger]DurableOrchestrationContext context)
{
    var person = context.GetInput<Person>();
    var firstNameError = await context.CallActivityAsync<Error>(nameof(CheckFirstName), person);

    return new[] { firstNameError };
}

A few things happened here:

1. We had to make the method async because we're executing CallActivityAsync to kick off the activity function
2. Our orchestration returns a list of Errors, like our API expects, so the return value becomes Task<IList<Error>>
3. We get the input to send to our activity from the input that came in to the orchestrator; namely the Person object we deserialized in the starter.

At this point our DF is fully testable. Let's give it a run and observe the output:

[5/31/2018 4:09:00 PM] Generating 3 job function(s)
[5/31/2018 4:09:00 PM] Found the following functions:
[5/31/2018 4:09:00 PM] Functions.Function1.RunAsync
[5/31/2018 4:09:00 PM] Functions.Function1.Start
[5/31/2018 4:09:00 PM] Functions.Function1.CheckFirstName

...

Http Functions:

        Validate: https://localhost:7071/api/Validate

Note that the host found 3 functions - good - but only exposed one via HTTP. That's because the other 2 are used only internally by DF to conduct the orchestration; an outside party has no business invoking them. Awesome. Let's kick off /Validate and see what happens:

POST /api/Validate HTTP/1.1
Host: localhost:7071
Content-Type: application/json
Cache-Control: no-cache
Postman-Token: 64700bdc-69d4-4faa-abc2-6628f5790715

{
    "Name" : {
        "First" : "J",
        "Last" : "Doe",
        "Title" : "Mr"
    },
    "Address" : {
        "Line1" : "1234 Anywhere St.",
        "Line2" : null,
        "City" : "Somewhere",
        "State" : "OR",
        "Zip" : "12345",
        "Country" : "United States of America"
    }
}

Which kicks off our DF execution and the host starts spitting out log data:

[5/31/2018 4:20:30 PM] Executing 'Validate' (Reason='This function was programmatically called via the host APIs.', Id=b2969100-d67e-4725-9545-7c017ff19923)
[5/31/2018 4:20:30 PM] Person request received:
[5/31/2018 4:20:30 PM] {
[5/31/2018 4:20:30 PM]     "Name" : {
[5/31/2018 4:20:30 PM]         "First" : "J",
[5/31/2018 4:20:30 PM]         "Last" : "Doe",
[5/31/2018 4:20:30 PM]         "Title" : "Mr"
[5/31/2018 4:20:30 PM]     },
[5/31/2018 4:20:30 PM]     "Address" : {
[5/31/2018 4:20:30 PM]         "Line1" : "1234 Anywhere St.",
[5/31/2018 4:20:30 PM]         "Line2" : null,
[5/31/2018 4:20:30 PM]         "City" : "Somewhere",
[5/31/2018 4:20:30 PM]         "State" : "OR",
[5/31/2018 4:20:30 PM]         "Zip" : "12345",
[5/31/2018 4:20:30 PM]         "Country" : "United States of America"
[5/31/2018 4:20:30 PM]     }
[5/31/2018 4:20:30 PM] }
[5/31/2018 4:20:30 PM] 35ec223e3c2f4aff91d7d9727a894a83: Function 'Start (Orchestrator)' scheduled. Reason: NewInstance. IsReplay: False. State: Scheduled. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 0.
[5/31/2018 4:20:34 PM] Executed 'Validate' (Succeeded, Id=b2969100-d67e-4725-9545-7c017ff19923)
[5/31/2018 4:20:37 PM] Executing 'Start' (Reason='', Id=4956f975-380b-4e75-86ec-bd86225512d2)
[5/31/2018 4:20:37 PM] 35ec223e3c2f4aff91d7d9727a894a83: Function 'Start (Orchestrator)' started. IsReplay: False. Input: (1188 bytes). State: Started. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 1.
[5/31/2018 4:20:37 PM] 35ec223e3c2f4aff91d7d9727a894a83: Function 'CheckFirstName (Activity)' scheduled. Reason: Start. IsReplay: False. State: Scheduled. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 2.
[5/31/2018 4:20:37 PM] Executed 'Start' (Succeeded, Id=4956f975-380b-4e75-86ec-bd86225512d2)
[5/31/2018 4:20:37 PM] 35ec223e3c2f4aff91d7d9727a894a83: Function 'Start (Orchestrator)' awaited. IsReplay: False. State: Awaited. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 3.
[5/31/2018 4:20:39 PM] 35ec223e3c2f4aff91d7d9727a894a83: Function 'CheckFirstName (Activity)' started. IsReplay: False. Input: (1196 bytes). State: Started. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 4.
[5/31/2018 4:20:39 PM] Executing 'CheckFirstName' (Reason='', Id=2ddb9093-2588-489e-a39e-a83a266971d7)
[5/31/2018 4:20:39 PM] Message received: 35ec223e3c2f4aff91d7d9727a894a83
[5/31/2018 4:20:39 PM]  - Error found: First name is null or not longer than 1 character
[5/31/2018 4:20:39 PM] Executed 'CheckFirstName' (Succeeded, Id=2ddb9093-2588-489e-a39e-a83a266971d7)
[5/31/2018 4:20:39 PM] 35ec223e3c2f4aff91d7d9727a894a83: Function 'CheckFirstName (Activity)' completed. ContinuedAsNew: False. IsReplay: False. Output: (280 bytes). State: Completed. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 5.
[5/31/2018 4:20:40 PM] Executing 'Start' (Reason='', Id=9d968ad5-6d88-4d9b-ba99-c266a0adbac9)
[5/31/2018 4:20:40 PM] 35ec223e3c2f4aff91d7d9727a894a83: Function 'Start (Orchestrator)' started. IsReplay: True. Input: (1188 bytes). State: Started. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 6.
[5/31/2018 4:20:40 PM] 35ec223e3c2f4aff91d7d9727a894a83: Function 'CheckFirstName (Activity)' scheduled. Reason: Start. IsReplay: True. State: Scheduled. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 7.
[5/31/2018 4:20:40 PM] 35ec223e3c2f4aff91d7d9727a894a83: Function 'Start (Orchestrator)' completed. ContinuedAsNew: False. IsReplay: False. Output: (288 bytes). State: Completed. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 8.
[5/31/2018 4:20:40 PM] Executed 'Start' (Succeeded, Id=9d968ad5-6d88-4d9b-ba99-c266a0adbac9)

After all is said and done, Postman gets this back for a response:

Content-Length →788
Content-Type →application/json; charset=utf-8
Date →Thu, 31 May 2018 16:20:34 GMT
Location →https://localhost:7071/runtime/webhooks/DurableTaskExtension/instances/35ec223e3c2f4aff91d7d9727a894a83?taskHub=DurableFunctionsHub&connection=Storage&code=oGDC2qyC2H8gWrNUjpHAZxgj8CiF44hLxN5nebJZmAbCDo6nouIxag==
Retry-After →10
Server →Kestrel

{
    "id": "35ec223e3c2f4aff91d7d9727a894a83",
    "statusQueryGetUri": "https://localhost:7071/runtime/webhooks/DurableTaskExtension/instances/35ec223e3c2f4aff91d7d9727a894a83?taskHub=DurableFunctionsHub&connection=Storage&code=oGDC2qyC2H8gWrNUjpHAZxgj8CiF44hLxN5nebJZmAbCDo6nouIxag==",
    "sendEventPostUri": "https://localhost:7071/runtime/webhooks/DurableTaskExtension/instances/35ec223e3c2f4aff91d7d9727a894a83/raiseEvent/{eventName}?taskHub=DurableFunctionsHub&connection=Storage&code=oGDC2qyC2H8gWrNUjpHAZxgj8CiF44hLxN5nebJZmAbCDo6nouIxag==",
    "terminatePostUri": "https://localhost:7071/runtime/webhooks/DurableTaskExtension/instances/35ec223e3c2f4aff91d7d9727a894a83/terminate?reason={text}&taskHub=DurableFunctionsHub&connection=Storage&code=oGDC2qyC2H8gWrNUjpHAZxgj8CiF44hLxN5nebJZmAbCDo6nouIxag=="
}

which seems confusing, but if you look at the Location header and punch that in to Postman as an HTTP GET request you receive:

{
    "runtimeStatus": "Completed",
    "input": {
        "$type": "Functions.Person, Functions",
        "Name": {
            "$type": "Functions.Name, Functions",
            "First": "J",
            "Last": "Doe",
            "Title": "Mr"
        },
        "Address": {
            "$type": "Functions.Address, Functions",
            "Line1": "1234 Anywhere St.",
            "Line2": null,
            "City": "Somewhere",
            "State": "OR",
            "Zip": "12345",
            "Country": "United States of America"
        }
    },
    "customStatus": null,
    "output": [
        {
            "id": 1,
            "message": "First name is null or not longer than 1 character"
        }
    ],
    "createdTime": "2018-05-31T16:20:30Z",
    "lastUpdatedTime": "2018-05-31T16:20:40Z"
}

and have a look at that output property - huzzah!

For more information on the URLs given to you by the response from a POST to a Durable Functions starter, you can read the docs here.

Let's evolve our workflow, though, in to something more complex by first "chaining" our functions together, and then running them in parallel (fan out/in)

Function chaining with Durable Functions

As you might've guessed by now, you can chain functions together by simply coordinating their inputs and outputs. If we add in the Last Name validation to our flow, we end up with a new Activity function:

[FunctionName(nameof(CheckLastName))]
public static async Task<InputOutput> CheckLastName([ActivityTrigger]DurableActivityContext context, TraceWriter log)
{
    await Task.Delay(1400);

    var io = context.GetInput<InputOutput>();
    if ((io?.Input?.Name?.Last?.Length > 1) == false)
    {
        io.Output.Add(new Error { id = 2, message = "Last name is null or not longer than 1 character" });
    }

    return io;
}

Here we've set up the input to our 2nd function to be and InputOutput type that looks like this:

public class InputOutput
{
    public IList<Error> Output { get; set; } = new Error[0];

    public Person Input { get; set; }
}

So each function can augment it as they see fit, and at the end of the orchestration we'll simply store Output as the result.

This changes our implementation of the Starter and CheckFirstName as well, but only slightly:

[FunctionName(nameof(CheckFirstName))]
public static async Task<InputOutput> CheckFirstName([ActivityTrigger]DurableActivityContext context, TraceWriter log)
{
    await Task.Delay(1300);
    var person = context.GetInput<Person>();
    log.Info($@"Message received: {context.InstanceId}");
    if ((person?.Name?.First?.Length > 1) == false)
    {
        var err = new Error { id = 1, message = "First name is null or not longer than 1 character" };
        log.Info($@" - Error found: {err.message}");
        return new InputOutput { Output = new[] { err }, Input = person };
    }
    else
    {
        log.Info($@" - No error found");
        return new InputOutput { Input = person };
    }
}

[FunctionName(nameof(Start))]
public static async Task<IList<Error>> Start([OrchestrationTrigger]DurableOrchestrationContext context)
{
    var person = context.GetInput<Person>();

    var chainedInputOutput = await context.CallActivityAsync<InputOutput>(nameof(CheckFirstName), person);
    chainedInputOutput = await context.CallActivityAsync<InputOutput>(nameof(CheckLastName), chainedInputOutput);

    return chainedInputOutput.Output;
}

Let's give it a run and observe the output from our Functions host:

[5/31/2018 4:36:06 PM] Executing 'Validate' (Reason='This function was programmatically called via the host APIs.', Id=ba7a080e-a701-451f-971b-d7ea9670ce60)
[5/31/2018 4:36:07 PM] Person request received:
[5/31/2018 4:36:07 PM] {
[5/31/2018 4:36:07 PM]     "Name" : {
[5/31/2018 4:36:07 PM]         "First" : "J",
[5/31/2018 4:36:07 PM]         "Last" : "Doe",
[5/31/2018 4:36:07 PM]         "Title" : "Mr"
[5/31/2018 4:36:07 PM]     },
[5/31/2018 4:36:07 PM]     "Address" : {
[5/31/2018 4:36:07 PM]         "Line1" : "1234 Anywhere St.",
[5/31/2018 4:36:07 PM]         "Line2" : null,
[5/31/2018 4:36:07 PM]         "City" : "Somewhere",
[5/31/2018 4:36:07 PM]         "State" : "OR",
[5/31/2018 4:36:07 PM]         "Zip" : "12345",
[5/31/2018 4:36:07 PM]         "Country" : "United States of America"
[5/31/2018 4:36:07 PM]     }
[5/31/2018 4:36:07 PM] }
[5/31/2018 4:36:07 PM] ebc3b429c89942429046ce54c54eaccb: Function 'Start (Orchestrator)' scheduled. Reason: NewInstance. IsReplay: False. State: Scheduled. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 0.
[5/31/2018 4:36:09 PM] Executed 'Validate' (Succeeded, Id=ba7a080e-a701-451f-971b-d7ea9670ce60)
[5/31/2018 4:36:10 PM] Executing 'Start' (Reason='', Id=4cfd7beb-89f8-4b84-b738-a47b1d6e8c13)
[5/31/2018 4:36:10 PM] ebc3b429c89942429046ce54c54eaccb: Function 'Start (Orchestrator)' started. IsReplay: False. Input: (1188 bytes). State: Started. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 1.
[5/31/2018 4:36:10 PM] ebc3b429c89942429046ce54c54eaccb: Function 'CheckFirstName (Activity)' scheduled. Reason: Start. IsReplay: False. State: Scheduled. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 2.
[5/31/2018 4:36:10 PM] Executed 'Start' (Succeeded, Id=4cfd7beb-89f8-4b84-b738-a47b1d6e8c13)
[5/31/2018 4:36:10 PM] ebc3b429c89942429046ce54c54eaccb: Function 'Start (Orchestrator)' awaited. IsReplay: False. State: Awaited. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 3.
[5/31/2018 4:36:13 PM] ebc3b429c89942429046ce54c54eaccb: Function 'CheckFirstName (Activity)' started. IsReplay: False. Input: (1196 bytes). State: Started. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 4.
[5/31/2018 4:36:13 PM] Executing 'CheckFirstName' (Reason='', Id=6a1dbbe1-2ac6-4a6e-84c4-3e506d94615b)
[5/31/2018 4:36:14 PM] Message received: ebc3b429c89942429046ce54c54eaccb
[5/31/2018 4:36:14 PM]  - Error found: First name is null or not longer than 1 character
[5/31/2018 4:36:15 PM] Executed 'CheckFirstName' (Succeeded, Id=6a1dbbe1-2ac6-4a6e-84c4-3e506d94615b)
[5/31/2018 4:36:15 PM] ebc3b429c89942429046ce54c54eaccb: Function 'CheckFirstName (Activity)' completed. ContinuedAsNew: False. IsReplay: False. Output: (1104 bytes). State: Completed. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 5.
[5/31/2018 4:36:16 PM] Executing 'Start' (Reason='', Id=ccae9c2c-4243-48f1-bbee-5926687350fc)
[5/31/2018 4:36:16 PM] ebc3b429c89942429046ce54c54eaccb: Function 'Start (Orchestrator)' started. IsReplay: True. Input: (1188 bytes). State: Started. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 6.
[5/31/2018 4:36:16 PM] ebc3b429c89942429046ce54c54eaccb: Function 'CheckFirstName (Activity)' scheduled. Reason: Start. IsReplay: True. State: Scheduled. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 7.
[5/31/2018 4:36:16 PM] ebc3b429c89942429046ce54c54eaccb: Function 'CheckLastName (Activity)' scheduled. Reason: Start. IsReplay: False. State: Scheduled. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 8.
[5/31/2018 4:36:16 PM] Executed 'Start' (Succeeded, Id=ccae9c2c-4243-48f1-bbee-5926687350fc)
[5/31/2018 4:36:16 PM] ebc3b429c89942429046ce54c54eaccb: Function 'Start (Orchestrator)' awaited. IsReplay: False. State: Awaited. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 9.
[5/31/2018 4:36:18 PM] ebc3b429c89942429046ce54c54eaccb: Function 'CheckLastName (Activity)' started. IsReplay: False. Input: (1964 bytes). State: Started. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 10.
[5/31/2018 4:36:18 PM] Executing 'CheckLastName' (Reason='', Id=aff38799-8268-435b-934c-d94dff88d931)
[5/31/2018 4:36:20 PM] Executed 'CheckLastName' (Succeeded, Id=aff38799-8268-435b-934c-d94dff88d931)
[5/31/2018 4:36:20 PM] ebc3b429c89942429046ce54c54eaccb: Function 'CheckLastName (Activity)' completed. ContinuedAsNew: False. IsReplay: False. Output: (1104 bytes). State: Completed. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 11.
[5/31/2018 4:36:21 PM] Executing 'Start' (Reason='', Id=e7f8047e-eccc-4a75-aab0-9ac085f03b72)
[5/31/2018 4:36:21 PM] ebc3b429c89942429046ce54c54eaccb: Function 'Start (Orchestrator)' started. IsReplay: True. Input: (1188 bytes). State: Started. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 12.
[5/31/2018 4:36:21 PM] ebc3b429c89942429046ce54c54eaccb: Function 'CheckFirstName (Activity)' scheduled. Reason: Start. IsReplay: True. State: Scheduled. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 13.
[5/31/2018 4:36:21 PM] ebc3b429c89942429046ce54c54eaccb: Function 'CheckFirstName (Activity)' completed. ContinuedAsNew: False. IsReplay: True. Output: (replayed). State: Completed. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 14.
[5/31/2018 4:36:21 PM] ebc3b429c89942429046ce54c54eaccb: Function 'CheckLastName (Activity)' scheduled. Reason: Start. IsReplay: True. State: Scheduled. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 15.
[5/31/2018 4:36:21 PM] ebc3b429c89942429046ce54c54eaccb: Function 'Start (Orchestrator)' completed. ContinuedAsNew: False. IsReplay: False. Output: (288 bytes). State: Completed. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 16.
[5/31/2018 4:36:22 PM] Executed 'Start' (Succeeded, Id=e7f8047e-eccc-4a75-aab0-9ac085f03b72)

Notice in our activity functions I had added a couple of await Task.Delay calls. Also notice that our full execution took around 15 seconds, but in Postman I had a response back from the endpoint within 5.

A subsequent call to the Location header value of the 202 ACCEPTED response I got back yielded:

{
    "runtimeStatus": "Completed",
    "input": {
        "$type": "Functions.Person, Functions",
        "Name": {
            "$type": "Functions.Name, Functions",
            "First": "J",
            "Last": "Doe",
            "Title": "Mr"
        },
        "Address": {
            "$type": "Functions.Address, Functions",
            "Line1": "1234 Anywhere St.",
            "Line2": null,
            "City": "Somewhere",
            "State": "OR",
            "Zip": "12345",
            "Country": "United States of America"
        }
    },
    "customStatus": null,
    "output": [
        {
            "id": 1,
            "message": "First name is null or not longer than 1 character"
        }
    ],
    "createdTime": "2018-05-31T16:36:07Z",
    "lastUpdatedTime": "2018-05-31T16:36:22Z"
}

exactly what we expected.

What happens if we call the location endpoint before the run's completed? Get your ninja copy/paste skills ready and give it a shot:

{
    "runtimeStatus": "Running",
    "input": {
        "$type": "Functions.Person, Functions",
        "Name": {
            "$type": "Functions.Name, Functions",
            "First": "J",
            "Last": "Doe",
            "Title": "Mr"
        },
        "Address": {
            "$type": "Functions.Address, Functions",
            "Line1": "1234 Anywhere St.",
            "Line2": null,
            "City": "Somewhere",
            "State": "OR",
            "Zip": "12345",
            "Country": "United States of America"
        }
    },
    "customStatus": null,
    "output": null,
    "createdTime": "2018-05-31T16:39:22Z",
    "lastUpdatedTime": "2018-05-31T16:39:28Z"
}

notice the first property, runtimeStatus with a value of Running and how our output property is still null. This indicates the DF is in progress and the caller should check back later. Waiting a few more seconds to call it and we get back

{
    "runtimeStatus": "Completed",
    "input": {
        "$type": "Functions.Person, Functions",
        "Name": {
            "$type": "Functions.Name, Functions",
            "First": "J",
            "Last": "Doe",
            "Title": "Mr"
        },
        "Address": {
            "$type": "Functions.Address, Functions",
            "Line1": "1234 Anywhere St.",
            "Line2": null,
            "City": "Somewhere",
            "State": "OR",
            "Zip": "12345",
            "Country": "United States of America"
        }
    },
    "customStatus": null,
    "output": [
        {
            "id": 1,
            "message": "First name is null or not longer than 1 character"
        }
    ],
    "createdTime": "2018-05-31T16:39:22Z",
    "lastUpdatedTime": "2018-05-31T16:39:37Z"
}

as expected.

So now you can see that chaining functions, or executing them sequentially is as simple as async/await elsewhere in .Net. But what about running them in parallel (aka Fan out/in pattern)?

Parallel execution (Fan out/in pattern)

Well, believe it or not you've already seen how this is done, it's just got a bit of a DF twist on it. Much like our Azure function, the trick here is to put all the CallActivityAsync<> calls in to a Task[] collection and then WaitAll on the collection. As you also might have figured out, our current InputOutput construct won't work here due to the parallel nature so instead let's just have each function put out its error (or null) and we'll aggregate when we're done.

Let's get started.

First, change our Starter to call the activities in parallel and wait on the collection:

var tasks = new[]
{
    context.CallActivityAsync<Error>(nameof(CheckFirstName), person),
    context.CallActivityAsync<Error>(nameof(CheckLastName), person)
};

var errors = await Task.WhenAll(tasks);

You see, also, here that we're now just giving our Person object to everybody, which changes our implementation of the activities slightly:

[FunctionName(nameof(CheckFirstName))]
public static async Task<Error> CheckFirstName([ActivityTrigger]DurableActivityContext context, TraceWriter log)
{
    await Task.Delay(1300);
    var person = context.GetInput<Person>();
    log.Info($@"Message received: {context.InstanceId}");
    if ((person?.Name?.First?.Length > 1) == false)
    {
        var err = new Error { id = 1, message = "First name is null or not longer than 1 character" };
        log.Info($@" - Error found: {err.message}");
        return err;
    }

    log.Info($@" - No error found");
    return null;
}

[FunctionName(nameof(CheckLastName))]
public static async Task<Error> CheckLastName([ActivityTrigger]DurableActivityContext context, TraceWriter log)
{
    await Task.Delay(1400);

    var person = context.GetInput<Person>();
    if ((person?.Name?.Last?.Length > 1) == false)
    {
        var err = new Error { id = 2, message = "Last name is null or not longer than 1 character" };
        log.Info($@" - Error found: {err.message}");
        return err;
    }

    log.Info($@" - No error found");
    return null;
}

And finally, we need to return the results from our orchestrator:

var errors = await Task.WhenAll(tasks);

return errors.Where(e => e != null).ToList();

Let's give it a run, paying attention to the output as we should see both Activity Functions executing simultaneously. And sure enough:

[5/31/2018 4:49:13 PM] Executing 'Validate' (Reason='This function was programmatically called via the host APIs.', Id=4496dbf0-d09e-44fc-95d2-2445ea612bbb)
[5/31/2018 4:49:13 PM] Person request received:
[5/31/2018 4:49:13 PM] {
[5/31/2018 4:49:13 PM]     "Name" : {
[5/31/2018 4:49:13 PM]         "First" : "J",
[5/31/2018 4:49:13 PM]         "Last" : "Doe",
[5/31/2018 4:49:13 PM]         "Title" : "Mr"
[5/31/2018 4:49:13 PM]     },
[5/31/2018 4:49:13 PM]     "Address" : {
[5/31/2018 4:49:13 PM]         "Line1" : "1234 Anywhere St.",
[5/31/2018 4:49:13 PM]         "Line2" : null,
[5/31/2018 4:49:13 PM]         "City" : "Somewhere",
[5/31/2018 4:49:13 PM]         "State" : "OR",
[5/31/2018 4:49:13 PM]         "Zip" : "12345",
[5/31/2018 4:49:13 PM]         "Country" : "United States of America"
[5/31/2018 4:49:13 PM]     }
[5/31/2018 4:49:13 PM] }
[5/31/2018 4:49:13 PM] 4e580cf9c1f841a6b820eb030e6c37bf: Function 'Start (Orchestrator)' scheduled. Reason: NewInstance. IsReplay: False. State: Scheduled. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 13.
[5/31/2018 4:49:14 PM] Executed 'Validate' (Succeeded, Id=4496dbf0-d09e-44fc-95d2-2445ea612bbb)
[5/31/2018 4:49:17 PM] Executing 'Start' (Reason='', Id=2c80b880-6062-4a04-a2c3-5b2f6fc0490e)
[5/31/2018 4:49:17 PM] 4e580cf9c1f841a6b820eb030e6c37bf: Function 'Start (Orchestrator)' started. IsReplay: False. Input: (1188 bytes). State: Started. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 14.
[5/31/2018 4:49:18 PM] 4e580cf9c1f841a6b820eb030e6c37bf: Function 'CheckFirstName (Activity)' scheduled. Reason: Start. IsReplay: False. State: Scheduled. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 15.
[5/31/2018 4:49:18 PM] 4e580cf9c1f841a6b820eb030e6c37bf: Function 'CheckLastName (Activity)' scheduled. Reason: Start. IsReplay: False. State: Scheduled. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 16.
[5/31/2018 4:49:18 PM] Executed 'Start' (Succeeded, Id=2c80b880-6062-4a04-a2c3-5b2f6fc0490e)
[5/31/2018 4:49:18 PM] 4e580cf9c1f841a6b820eb030e6c37bf: Function 'Start (Orchestrator)' awaited. IsReplay: False. State: Awaited. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 17.
[5/31/2018 4:49:19 PM] 4e580cf9c1f841a6b820eb030e6c37bf: Function 'CheckFirstName (Activity)' started. IsReplay: False. Input: (1196 bytes). State: Started. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 18.
[5/31/2018 4:49:19 PM] Executing 'CheckFirstName' (Reason='', Id=c1607f15-607d-40e1-a14e-74a1c0046c13)
[5/31/2018 4:49:20 PM] 4e580cf9c1f841a6b820eb030e6c37bf: Function 'CheckLastName (Activity)' started. IsReplay: False. Input: (1196 bytes). State: Started. HubName: DurableFunctionsHub. AppName: . SlotName: . ExtensionVersion: 1.4.1.0. SequenceNumber: 19.
[5/31/2018 4:49:20 PM] Executing 'CheckLastName' (Reason='', Id=8a38bb91-4c3b-431a-bfb9-66e8c6f259bd)

...

Notice that CheckFirstName has not completed before CheckLastName kicked off. And yet, we waited for all to be done before our output value arrived at our endpoint and was populated correctly.

The implementation

So how is Durable Functions managing all this? It's all done within the storage account you specify for AzureWebJobsStorage; required by Azure Functions. If you open Storage Explorer and have a look, you'll see a number of Durable Functions-created containers, tables, and queues:

and it is within these that DF manages the state of each orchestration including inputs/outputs to/from Activity functions and where each orchestration instance is in its execution.

In short, DF manages execution by using Storage Queues to pass messages to/from Activity Functions and the Orchestrator. The Orchestrator maintains the state of an individual orchestration within Table storage and that's what you're getting when you hit the Location endpoint returned by the 202 ACCEPTED response to the Starter.

Because Storage Queues only have polling capability today w/in Functions, this is also why this particular set up takes longer to run than the Service Bus or Logic Apps approach shown in Step 3. My hope for the future is this will eventually be driven by Event Grid for near-instantaneous execution of activities & orchestrations.

That concludes the series! I hope you've found this insightful and educational as you plan to move your workflows to the cloud and take advantage of the microbilling and autoscaling that Azure Serverless has to offer!

You've likely already heard of Azure Functions (otherwise hey click here!), Azure's serverless web execution allowing you to do thousands, even millions, of executions for merely pennies per run. In many cases you can get away with a serverless implementation completely free!

With that in mind, let's implement our solution from earlier in this series in Azure Functions. As a reminder here are the requirements:

When I receive a request to my Function, I want to

1. Validate the schema of the incoming JSON request.
2. Validate the content of the JSON request against business rules.
3. Return a response indicating which validations failed and which ones passed.

We've defined our schema as a Person schema like so:

{
    "Name" : {
        "First" : "John",
        "Last" : "Doe",
        "Title" : "Mr"
    },
    "Address" : {
        "Line1" : "1234 Anywhere St.",
        "Line2" : null,
        "City" : "Somewhere",
        "State" : "OR",
        "Zip" : "12345",
        "Country" : "United States of America"
    }
}

For our business rules, we'll enforce:

1. First & Last name must be present & longer than 1 character
2. If Address Line 2 is not null, Line 1 must also not be null
3. City must be present
4. State must be present & only a 2-letter abbreviation
5. Zip must be present and 5 characters

Step 1: Create an Azure Functions project

If you haven't already, install VS 2017 (Community Edition is fine) with the Azure Workload option. Next,

1. Click File | New Project
2. Choose Cloud | Azure Function
3. Choose Azure Functions v2, HTTP Trigger, Storage Emulator, Function authentication

Step 2: Validate the schema

Like most things, Visual Studio makes this cake with the 'Paste JSON as code' feature. Simply highlight the JSON above, go to VS 2017, create a new class file (let's call it Person.cs), click 'Edit' | 'Paste Special' | 'Paste JSON as classes' and voila:

namespace Functions
{
    public class Rootobject
    {
        public Name Name { get; set; }
        public Address Address { get; set; }
    }

    public class Name
    {
        public string First { get; set; }
        public string Last { get; set; }
        public string Title { get; set; }
    }

    public class Address
    {
        public string Line1 { get; set; }
        public object Line2 { get; set; }
        public string City { get; set; }
        public string State { get; set; }
        public string Zip { get; set; }
        public string Country { get; set; }
    }
}

for readability, rename Rootobject to Person. Because we put null in the sample for Line2, we need to fix that up by changing it to

public *string* Line2 { get; set; }

Back in Function1 now, we can enforce the validation of the incoming JSON by simply telling the HttpTrigger binding that it should be of type Person instead of HttpRequest:

public static IActionResult Run([HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]Person req, TraceWriter log)

Let's also clean this up by allowing only POST requests, not GET requests:

public static IActionResult Run([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]Person req, TraceWriter log)

and finally, let's name the function Validate:

[FunctionName("Validate")]
public static IActionResult Run([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]Person req, TraceWriter log)

Now our function will ensure any incoming POST request to /Validate has a body that can be deserialized to our Person object.

Step 3: Run the validations

"Chained" approach

Our validations are stored in an array (more than one can fail) so let's fab that code up now and see how to return it as part of an HTTP 200 response from the Function:

var errors = new List<Error>();

return new OkObjectResult(errors);

Where we've defined Error to be:

private class Error
{
    public int id { get; set; }
    public string message { get; set; }
};

Functions will, by default, return JSON payloads for all types output to the *ObjectResult types returned from a Function method.

Obviously the validations we have can be done in straight .Net pretty simply:

if ((req?.Name?.First?.Length > 1) == false)
{
    errors.Add(new Error { id = 1, message = "First name is null or not longer than 1 character" });
}

if ((req?.Name?.Last?.Length > 1) == false)
{
    errors.Add(new Error { id = 2, message = "Last name is null or not longer than 1 character" });
}

if (!string.IsNullOrEmpty(req?.Address?.Line2) && string.IsNullOrEmpty(req?.Address?.Line1))
{
    errors.Add(new Error { id = 3, message = "Address line 2 is populated but line 1 is empty" });
}

if (string.IsNullOrEmpty(req?.Address?.City))
{
    errors.Add(new Error { id = 4, message = "City is empty" });
}

if ((req?.Address?.State?.Length == 2) == false)
{
    errors.Add(new Error { id = 5, message = "State is null or not a 2-letter abbreviation" });
}

if ((req?.Address?.Zip?.Length == 5) == false)
{
    errors.Add(new Error { id = 6, message = "Zip is null or not 5 digits" });
}

and it works splendidly:

POST /api/Validate HTTP/1.1
Host: localhost:7071
Content-Type: application/json
Cache-Control: no-cache
Postman-Token: db88214a-8699-4766-bd48-d1a6d1fd495b

{
    "Name" : {
        "First" : "J",
        "Last" : "Doe",
        "Title" : "Mr"
    },
    "Address" : {
        "Line1" : "1234 Anywhere St.",
        "Line2" : null,
        "City" : "Somewhere",
        "State" : "OR",
        "Zip" : "12345",
        "Country" : "United States of America"
    }
}

Content-Type →application/json; charset=utf-8
Date →Wed, 30 May 2018 04:30:29 GMT
Server →Kestrel
Transfer-Encoding →chunked

[
    {
        "id": 1,
        "message": "First name is null or not longer than 1 character"
    }
]

Fan out/in approach

The previous code roughly corresponds to the sequential Flow set up in the first post of this series. But let's be real, we'd most certainly be doing something a bit more complex than these operations in real life, right? So let's async this work, and go on to extend it even further.

The body of our Function becomes:

var errors = new List<Error>();

// Fan Out
var checks = new List<Task<Error>>
{
    CheckFirstName(req, errors),
    CheckLastName(req, errors),
    CheckAddressLines(req, errors),
    CheckCity(req, errors),
    CheckState(req, errors),
    CheckZip(req, errors)
};

// wait for all checks to complete (Fan In)
await Task.WhenAll(checks);

// Add any non-null values to our errors list
errors.AddRange(checks.Select(t => t.Result).Where(r => r != null));

return new OkObjectResult(errors);

In this approach we kick off all the Check* methods to do their work and add the resulting Task<Error> object to a collection which we then wait for all tasks to complete. Once they do, any that have a non-null result have returned an Error object and we should add that to our list to return back to the caller as the JSON body of our HTTP 200 response.

Each Check* method looks like this:

private static async Task<Error> CheckZip(Person req)
{
    await Task.Delay(1000);
    if ((req?.Address?.Zip?.Length == 5) == false)
    {
        return new Error { id = 6, message = "Zip is null or not 5 digits" };
    }

    return null;
}

private static async Task<Error> CheckState(Person req)
{
    await Task.Delay(1100);
    if ((req?.Address?.State?.Length == 2) == false)
    {
        return new Error { id = 5, message = "State is null or not a 2-letter abbreviation" };
    }

    return null;
}

private static async Task<Error> CheckCity(Person req)
{
    await Task.Delay(1200);
    if (string.IsNullOrEmpty(req?.Address?.City))
    {
        return new Error { id = 4, message = "City is empty" };
    }

    return null;
}

private static async Task<Error> CheckAddressLines(Person req)
{
    await Task.Delay(1300);
    if (!string.IsNullOrEmpty(req?.Address?.Line2) && string.IsNullOrEmpty(req?.Address?.Line1))
    {
        return new Error { id = 3, message = "Address line 2 is populated but line 1 is empty" };
    }

    return null;
}

private static async Task<Error> CheckLastName(Person req)
{
    await Task.Delay(1400);
    if ((req?.Name?.Last?.Length > 1) == false)
    {
        return new Error { id = 2, message = "Last name is null or not longer than 1 character" };
    }

    return null;
}

private static async Task<Error> CheckFirstName(Person req)
{
    await Task.Delay(1500);
    if ((req?.Name?.First?.Length > 1) == false)
    {
        return new Error { id = 1, message = "First name is null or not longer than 1 character" };
    }

    return null;
}

Notice that I've put in Task.Delay() calls to artificially make the execution of the methods take some time; emulating real-world behavior.

What if our calls were other Azure Functions, though? Well, not a whole lot changes except for the signatures of our validation methods and how we call them. Let's take a look:

[FunctionName("CheckZip")]
public static async Task<IActionResult> CheckZip([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]Person req)
{
    await Task.Delay(1000);
    if ((req?.Address?.Zip?.Length == 5) == false)
    {
        return new OkObjectResult(new Error { id = 6, message = "Zip is null or not 5 digits" });
    }

    return new NoContentResult();
}

We label the method as a Function, denote it's triggered via HTTP, and return an Http IActionResult instead of an object.

We also need to add an HttpClient to our Function:

private static HttpClient _client = new HttpClient { BaseAddress = new System.Uri(@"https://localhost:7071/api/") };

I set the BaseAddress here to make the PostAsync() calls cleaner later

With those changes made to all the Check methods, let's look at the body of our "orchestrator" and how it changes:

var errors = new List<Error>();

var personPostContent = new StringContent(JsonConvert.SerializeObject(req), Encoding.Default, @"application/json");
// Fan Out
var checks = new List<Task<HttpResponseMessage>>
{
    _client.PostAsync(@"CheckFirstName", personPostContent),
    _client.PostAsync(@"CheckLastName", personPostContent),
    _client.PostAsync(@"CheckAddressLines", personPostContent),
    _client.PostAsync(@"CheckCity", personPostContent),
    _client.PostAsync(@"CheckState", personPostContent),
    _client.PostAsync(@"CheckZip", personPostContent)
};

// wait for all checks to complete (Fan In)
var responses = await Task.WhenAll(checks);

// Add any non-null values to our errors list
foreach (var response in responses)
{
    var err = await response.Content.ReadAsAsync<Error>();
    if (err != null)
    {
        errors.Add(err);
    }
}

return new OkObjectResult(errors);

I wasn't able to get PostAsJsonAsync methods on HttpClient to work successfully; the payload kept coming to the Functions as null instead of being properly serialized/deserialized to a Person object. Hence the need for the personPostContent line.

Debugging the Function now, you see many Functions indexed by the host, and their execution all kicked off in parallel. But at the end of the day they all "fan back in" and we process their results and return it back to the caller.

There's are a couple of problems with this implementation, though.

1. We're now distributed across Azure Functions or even disparate web services. This means latency problems, connection issues, and the like.
2. Azure Functions, in the Consumption plan, is limited to 5-minutes per execution by default (you can increase it to 10 if you like with some host.json editing). What if our total orchestration takes longer than this?

How can we solve these?

Tackling it with Logic Apps

One way to handle this issue is to orchestrate our Function calls with Azure's purpose-build orchestration offering: Logic Apps. As you might expect, its integration with Functions is first-class; let's take a look.

First, deploy the Functions we wrote above (the orchestrator and all the sub-task functions) out to Azure so we can use them from our Logic App.

Next, create a new Logic App in Azure, and head to the designer surface.

The first step of our Logic App is going to be the same as it was for Flow (and the subsequent LA import); a 'Request' trigger with JSON schema in it matching what we expect from our caller. If you did the Microsoft Flow part of this series, it should look familiar when you're done:

Next, let's orchestrate the call to the first Function. Click 'Add an action' and type 'functions' in the search area. Choose 'Azure Function - Choose an Azure Function'

Next, choose the Azure Functions instance to which you deployed all the functions, and select the first of our validation functions:

In the request body for this function, you'll pass the same body that came in to the Logic App:

Using the same actions as before, we need to create a variable, initialize it to empty, then append the responses from each function on to it and, finally, send that back as the Response from the Logic App. By the time you're done, it should look like this:

"That's great" you might be wondering, "but how does this help the retry/reliability of my orchestration?"

For a glimpse in to this, click the '...' in the upper right of any of the Functions actions and choose 'Settings'

Notice the 'Timeout' and 'Retry Policy' options. Also make note of the 'Asynchronous pattern' handling built-in. With these 3 options, you can achieve extreme resilience in your orchestration. I encourage you to try them out as you experiment more with serverless workflow orchestration.

Logic Apps enforces a limit of 120s on any single HTTP-based action. So your orchestration steps must come back w/in that timeframe or LA will kill them.

Messaging approach

There's another way we could string our Azure Functions together, or fan out/in their execution, using Azure's Messaging offerings: Service Bus, Event Hubs, and Event Grid. For reliable orchestration, I recommend going with Service Bus as it provides queues with de-duplication, partitioning, and dead-lettering capabilities. In general, SB is recommended for "transactional" systems where delivery is critical. Since we wouldn't want any part of our orchestration to fall over, it fits here.

The usual design of this type of setup is to put "messages" for each Azure Function in to a Service Bus queue and set up each "worker" of the orchestration to pick off their own queue for work. This isn't too complex to set up, but the back side of our orchestration - fanning back in from a fan out design - can be. Consider this: How do you ensure that an execution that's part of a fan out comes back in to the same execution?

However, if you're not doing parallel execution, then the path is more simple; you just put things in queue A for worker A, then worker A puts them in queue B for worker B and so on until you're done and you put the result in a queue for the "receiver" which ends the orchestration.

In both cases, you will be making use of the "Asynchronous Pattern" shown off by Logic Apps where you return an HTTP 202 Accepted back to the caller with a 'Location' endpoint they can "ping" to get back the result. You can read more about the async pattern served by HTTP 202 here at the REST cookbook.

Azure Function Bindings make the "serial" orchestration really easy: the "starter" function usually binds to HttpTrigger and has a ServiceBus output binding. All subsequent functions have a ServiceBusTrigger and ServiceBus output binding. The final function has a ServiceBusTrigger and another output binding in which to "store" the final result. Optionally, this last function could output to another endpoint (callback, Event Grid, etc) with the final result.

Typically this approach is implemented as follows:

• "Start" Function receives a request
• Decorate the request with a unique ID (eg: GUID)
• For the "serial" execution path:
  • Queue the payload of the request (with new Unique ID) in to a Service Bus queue for Function 1
  • Function 1 does work, queues for Function 2
  • ...
  • Function N does work, drops result where /result/<id> will find it
• For the "parallel" (fan out/in) execution path
  • Enqueue the payload of the request (with new Unique ID) in to Service Bus queues for Functions 1...N
  • Function X does work on the payload, drops result where /result/<id> will find it
  • Important: In this case, a call to /result/<id> only returns once it has information from Functions 1...N available
• Send back 202 with Location of /result/<id>

The operation of the /result/<id> endpoint is up to you. I like to do something where it returns a 2xx code (but not 200) until the result is there, then returns 200 with the result when it's available. Ultimately the logic for what to do w/ the response from /result/<id> will be implemented by the caller that kicked off the Start function, so pick your poison here.

For the sake of brevity, I'll walk through coding up a Function that gets a request & drops the message, decorated with ID, in to a Service Bus queue and then another Function which picks up off the queue, does work, and drops the result in to Table Storage. Let's get started!

Step 1 - Create, of course, a Service Bus instance

Use the Standard tier for now.

Step 2 - Create a queue for each worker function in our workflow

It's up to you how you want to set up your queue but I would recommend enforcing duplicate detection and dead lettering upon expiration.

With dead-lettering on, the usual practice is to have a "watchdog" on the dead-letter box for any SB queue and do something with things that end up in there. You could re-queue them in to the same queue, log something, any number of things.

I've found sessions less-than-easy to work with inside Functions so, while they'd be something we would typically use for this approach (set the session id on the message to the unique id the orchestration function generates), I won't be using them in this sample.

Now we're ready for Step 3 - Set the Orchestrator to drop messages in to the queue

To get this started, let's take the Validate Function and get its bindings and return values set up for the procedure I outlined above.

First, add a [ServiceBus] parameter like so:

public static async Task<IActionResult> Run([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]Person req,
    [ServiceBus(@"function-a", Connection = @"ServiceBusOrchestrationConnection")]out QueueMessage functionAmsg,
    TraceWriter log)

The ServiceBus connection string must not contain any queue/topic information; it needs to be to the top level Service Bus instance or you'll get a runtime error. You can get this connection string by going to the Service Bus instance, clicking the RootManageSharedAccessKey, then copying either of the Connection Strings available. If you want to make your own Shared Access Key, that's fine too but it needs to have Manage permissions so the trigger can wire up as a listener and the output binding can send. Separate keys w/ these permissions are also acceptable.

What you'll immediately find is the binding for one parameter can only go to one queue. This means you either need to use multiple output parameters (eg: functionAmsg, functionBmsg, functionCmsg ...) or forego using the bindings altogether and just use a ServiceBus client object in your Function; I'll leave that up to you.

The other thing you'll notice is a new type, QueueMessage, that I've injected here. I did this so I had a place to put the unique id I'll assign the messages for retrieval later. This class is simple, and looks like this:

public class QueueMessage
{
    public string id { get; set; }
    public Person data { get; set; }
}

Now, when a new request comes in to the trigger (orchestrator), it needs to create a new guid, put it in the id property of a new QueueMessage object, and send that to our ServiceBus instance. Thanks to our bindings, this is just a handful of code:

var id = Guid.NewGuid().ToString();
log.Info($@"New orchestration started: {id}");
functionAmsg = new QueueMessage { id = id, data = req };

and off it goes.

But we also want to give back an HTTP 202 ACCEPTED response with a new function URL the caller can use to get the result

return new AcceptedResult($@"/api/GetResult?id={id}", functionAmsg);

does the trick nicely (we'll implement this later);

Next, let's configure the CheckFirstName Function to get messages from the new ServiceBus queue we've set up. In my configuration I called it function-a.

This changes our HttpTrigger for this function to ServiceBusTrigger and looks like so:

[FunctionName("CheckFirstName")]
public static IActionResult CheckFirstName([ServiceBusTrigger(@"function-a", Connection = @"ServiceBusOrchestrationConnection")]QueueMessage msg,
    TraceWriter log)
{
    if ((msg?.data?.Name?.First?.Length > 1) == false)
    {
        // store the error result
    }
    else
    {
        // store an empty error result
    }
}

For storage, we could choose to spin up a CosmosDB instance and store the JSON as documents there, or we could use the Table Storage that our Azure Functions already require (check out the AzureWebJobsStorage app setting). Functions makes both of these equally easy thanks to bindings, but for simplicity let's just go w/ the Table Storage we already have as part of our Functions.

To configure this as output for our CheckFirstName Function, change its signature to this:

public static void CheckFirstName([ServiceBusTrigger(@"function-a", Connection = @"ServiceBusOrchestrationConnection")]QueueMessage msg,
    [Table(@"checkfirstnameoutput", Connection = @"AzureWebJobsStorage")]out ErrorTableEntity err,
    TraceWriter log)

Notice we had to create a type that inherits from ITableEntity in order to use this. Our new ErrorTableEntity type simply wraps up our previous one:

public class ErrorTableEntity : TableEntity
{
    public ErrorTableEntity() { }

    public ErrorTableEntity(Error err, string sessionId)
    {
        this.PartitionKey = this.RowKey = sessionId;

        this.Error = err;
    }

    public Error Error { get; set; }
}

With this we can now fill out the rest of our CheckFirstName Function:

log.Info($@"Message received: {msg.id}");
if ((msg?.data?.Name?.First?.Length > 1) == false)
{
    err = new ErrorTableEntity(new Error { id = 1, message = "First name is null or not longer than 1 character" }, msg.id);
    log.Info($@" - Error logged: {err.Error.message}");
}
else
{
    err = new ErrorTableEntity(new Error { id = 0 }, msg.id);
    log.Info($@" - NoError logged");
}

return new OkResult();

So at this point, with a request to our HTTP-triggered function we should see:

• The message gets decorated with a new GUID
• It's put in to the pipeline for our CheckFirstName function
• CheckFirstName kicks off with that message
  • ... processes its validity
  • ... drops the appropriate error in Table Storage for our session

The logging messages we put in place will help trace things. If you haven't already, now's a good time to go download the Azure Storage Explorer so you can have a look at the table storage in the emulator (if that's what you're local.settings.json is pointing to) when we test this out next!

Let's give 'er a whirl and see what happens!

Postman sends:

POST /api/Validate HTTP/1.1
Host: localhost:7071
Content-Type: application/json
Cache-Control: no-cache
Postman-Token: b995b4d7-63fb-4b12-b021-4f469089c61c

{
    "Name" : {
        "First" : "J",
        "Last" : "Doe",
        "Title" : "Mr"
    },
    "Address" : {
        "Line1" : "1234 Anywhere St.",
        "Line2" : null,
        "City" : "Somewhere",
        "State" : "OR",
        "Zip" : "12345",
        "Country" : "United States of America"
    }
}

And gets back:

Content-Type →application/json; charset=utf-8
Date →Thu, 31 May 2018 04:57:30 GMT
Location →/api/GetResult?id=d6b07382-3fdb-4f06-b9a9-ab0fbdf15594
Server →Kestrel
Transfer-Encoding →chunked

{
    "id": "d6b07382-3fdb-4f06-b9a9-ab0fbdf15594",
    "data": {
        "Name": {
            "First": "J",
            "Last": "Doe",
            "Title": "Mr"
        },
        "Address": {
            "Line1": "1234 Anywhere St.",
            "Line2": null,
            "City": "Somewhere",
            "State": "OR",
            "Zip": "12345",
            "Country": "United States of America"
        }
    }
}

And our Function spits out

[5/31/2018 5:04:50 AM] Executing 'Validate' (Reason='This function was programmatically called via the host APIs.', Id=39ff9db3-03b6-4252-8429-0bd30f4ed6a1)
[5/31/2018 5:04:50 AM] New orchestration started: d6b07382-3fdb-4f06-b9a9-ab0fbdf15594
[5/31/2018 5:04:51 AM] Executed 'Validate' (Succeeded, Id=39ff9db3-03b6-4252-8429-0bd30f4ed6a1)
[5/31/2018 5:04:51 AM] Executing 'CheckFirstName' (Reason='New ServiceBus message detected on 'function-a'.', Id=9665e73b-f945-48df-82a5-8258d45b4787)
[5/31/2018 5:04:51 AM] Message received: d6b07382-3fdb-4f06-b9a9-ab0fbdf15594
[5/31/2018 5:04:54 AM]  - Error logged: First name is null or not longer than 1 character
[5/31/2018 5:04:55 AM] Executed 'CheckFirstName' (Succeeded, Id=9665e73b-f945-48df-82a5-8258d45b4787)

And if we take a look in our Storage account:

A row for our session, with its output! Now we can write our GetResult function like this:

[FunctionName("GetResult")]
public static async Task<IActionResult> GetResultAsync([HttpTrigger(AuthorizationLevel.Function, "get", Route = null)]HttpRequest req, [Table(@"checkfirstnameoutput", Connection = @"AzureWebJobsStorage")]CloudTable firstnameoutputTable)
{
    string targetId = null;

    if (req.GetQueryParameterDictionary()?.TryGetValue(@"id", out targetId) == true)
    {
        var queryResult = await firstnameoutputTable.ExecuteAsync(TableOperation.Retrieve<ErrorTableEntity>(targetId, targetId));
        if (queryResult.Result != null)
        {
            return new OkObjectResult(queryResult.Result);
        }

        return new NotFoundResult();
    }
    else
    {
        return new BadRequestObjectResult(@"'id' parameter is required");
    }
}

And just like that:

GET /api/GetResult?id=d6b07382-3fdb-4f06-b9a9-ab0fbdf15594 HTTP/1.1
Host: localhost:7071
Content-Type: application/json
Cache-Control: no-cache
Postman-Token: cced7ca9-a039-43d9-ae30-6c7d14c306de

Gets back

{
    "ErrorId": 1,
    "ErrorMessage": "First name is null or not longer than 1 character",
    "PartitionKey": "d6b07382-3fdb-4f06-b9a9-ab0fbdf15594",
    "RowKey": "d6b07382-3fdb-4f06-b9a9-ab0fbdf15594",
    "Timestamp": "2018-05-31T05:04:55.297+00:00",
    "ETag": "W/\"datetime'2018-05-31T05%3A04%3A55.297Z'\""
}

So we now have:

• An orchestration function delegating work to one or more workers and returning back to the caller where it can find the results
• A "lookup" Function to return results based on the session ID created by the orchestration function
• One or more worker functions each placing its ending state in the location where the lookup function can find it

If you were to do a Fan out/in process using this same architecture, your lookup function would - instead of checking one spot for output - handle the work of checking to make sure all applicable Functions have placed their stuff where it needs to be, or comes back with 404/204 to let the caller know "not done yet".

Tune in next time for the final part in the series where I cover a handy library shipped by a group within the Azure Functions team, Durable Functions, and how it helps abstract some of this work away for you!

1. Read & work through Part 1 of this series
2. When you're done, go to your Flow dashboard and click the Flow you just created
3. Click ... More near the top of the screen
4. Choose 'Export' | 'Logic Apps template (.json)'

DONE!

Ok not quite, let's walk through importing this in to Azure's IT-Pro Enterprise-grade workflow orchestration offering: Logic Apps. As of the time of this writing it's pretty well known in the integration/orchestration space that MSIT itself has migrated over 1000 backend processes at Microsoft from various systems, scripts, etc in to Logic Apps. This product is no joke, y'all. The execution engine for Logic Apps is the exact same one you're sending requests to when you submit a Azure resource deployment request (ARM Template, Portal, or otherwise); it's robust, resilient, and reliable.

Importing a Microsoft Flow in to Logic Apps

First thing's first, fire up your Azure Portal.

Now, because what you've exported is essentially the JSON definition for a Logic App that will do everything your Flow did, we need to deploy it as such. To do this in the portal, click "Create a resource" and search for, aptly named, "Template deployment"

Go ahead and click Create New and choose 'Build your own template in the editor':

Choose, as you might've guessed, 'Load File' and pick your exported Flow JSON file. You'll get the JSON loaded in, not formatted, so you just have to trust it 😉 Click 'Save' and behold some more Azure magic.

The next screen asks you for the Azure resource Basics, but also asks you to fill out any custom parameters your Flow needs! In our case, it only needs to know the name of the Flow and its location - easy peasy:

Once deployment is complete, head to the created Logic App. First thing's first, we have to turn it on by clicking the 'Enable' button.

Once you've done that, go ahead and click the 'Logic App Designer'. You should be met with a familiar site:

Let's check out our HTTP endpoint - this URL will be different from the one in Flow - and give it a quick test to make sure it's all kosher:

And that's all there is to "growing up to Logic Apps" from Microsoft Flow. The designer experience is exactly the same if you want to start from scratch and, as I alluded to at the end of the Flow blog post, Logic Apps has a much richer DevOps story as you can deploy them from ARM Templates using Visual Studio Team Services and even author them within Visual Studio 2017 using the Logic Apps Extension.

Next up, checking out this same implementation with Azure Functions!

In this series we take an in-depth look at how you can orchestrate and coordinate processes throughout your system using a variety of offerings on Microsoft's Azure platform. You'll get an overview, pros, cons, and how-tos for each product.

Microsoft Flow stepped on the scene along with a public preview of Microsoft PowerApps back in April, 2016. Primarily targeted at business users, trust me it can be so much more than just a tool to manage to get alerts from Sharepoint or your Inbox 😉

With the same suite of logical operators as Logic Apps (we'll talk more about these later), Flow can if/else, execute in parallel, loop, and retry with the best of them. And best of all, it has a free tier. Here's their current pricing model:

The best news? Everything you're about to see is possible within the Flow Free plan.

Let's get started with a problem statement: When I receive a request to my Flow, I want to

1. Validate the schema of the incoming JSON request.
2. Validate the content of the JSON request against business rules.
3. Return a response indicating which validations failed and which ones passed.

This seems pretty straightforward and something that might be reasonably common in any business environment. Put it on a /validate endpoint, send requests to it, and you've got a decoupled validation service. So let's see a few ways to skin this cat with Microsoft Flow:

Simple checks done in sequence

Step 1: Define our schema

Let's define our schema as a Person schema like so:

{
    "Name" : {
        "First" : "John",
        "Last" : "Doe",
        "Title" : "Mr"
    },
    "Address" : {
        "Line1" : "1234 Anywhere St.",
        "Line2" : null,
        "City" : "Somewhere",
        "State" : "OR",
        "Zip" : "12345",
        "Country" : "United States of America"
    }
}

For our business rules, we'll enforce:

1. First & Last name must be present & longer than 1 character
2. If Address Line 2 is not null, Line 1 must also not be null
3. City must be present
4. State must be present & only a 2-letter abbreviation
5. Zip must be present and 5 characters

And we'll return back a JSON that looks like:

{
    "Errors" : [
        {
            "id" : 1,
            "message" : "First name is empty or less than 1 character"
        },
        { 
            "id" : 3,
            "message" : "Address line 2 is not empty but line 1 is empty"
        },
        ...
    ]
}

Step 2: Make the Flow respond to an HTTP POST request containing a JSON body of a specific schema

This can be done with the HTTP - Request connector:

Upon choosing it, you'll notice something interesting (1):

Once you save your Flow, this area will be populated with the HTTP endpoint for you to hit. Simply copy it (2) and you're off to the races. For now, we'll use this step in the Flow to perform requirement #1: Validate the JSON schema.

Using the sample input shown prior, we can generate the JSON schema by clicking Use sample payload to generate schema (3) and pasting in the JSON body from above:

After clicking 'Done', you'll see a generated JSON schema show up in the HTTP - Request connector.

You can hand-type the JSON schema, but Flow won't validate that it's correct before Saving. You'll only know if you messed something up at runtime.

Step 3: Prep the return JSON value

Since we'll be executing a sequence of steps and putting the result for each validation in to a body to send back, we need to have a sort of "accumulator" variable. Remember we said ours would be an array. Lucky for us, Flow has just the thing.

Click 'New step' then 'Add an action'

type 'variable' in the search box and choose 'Initialize variable'

Next, name the variable and set it to be an Array type

Now we have an Array in to which we can add output from our validators as they run!

Step 4: Execute our first validation

First & Last name must be present & longer than 1 character

Let's tackle the First name, since this will just repeat for Last name.

Click 'New step' 'Add a condition' now:

Voila, you're met with a condition box, and spots to put both the "if true" and "if false" steps of your choosing. Perfect! For this check, the condition will simply be len(Name.First) > 1 (in rough pseudocode). Let's knock it out:

Click 'Choose a value' in the Condition box and note what happens. The JSON schema has been used to provide you an "intellisense" of sorts with the properties on the payload that are readily available for inspection. However, we need the length of one of these, not just the property value. So flip over to the 'Expressions' tab and notice what we have under 'Collections':

length() is exactly what we want. Click it. While the cursor is inside the (), flip back over to 'Dynamic content' and choose 'First'. You should end up with something like this:

Click 'OK' and fill out the rest of the condition:

If this fails, we want to append to our output array, so in the 'If no' area of our condition, click 'Add an action'. Type 'variable' in the search, and choose 'Append to array variable'. Add our first failure code:

{ "id" : 1, "message" : "First name is null or less than 2 characters in length" }

Since the rest will just be "rinse, repeat" let's give our Flow a run. Click 'Save' in the upper right and note that you now have a valid URL in our HTTP - Request trigger action at the top:

Heed the warning here! Since we've given a JSON schema to validate against, all requests coming in to our Flow must have Content-Type: application/json in their headers.

Fire up Postman and

1. Copy your URL in to a new request.
2. Switch the HTTP Method to POST
3. Add the Content-Type header
4. Fill out a raw body with the sample from the start of this post

Before you click 'Send' go to your Flow and click the 'Test' button in the upper right. Choose 'I'll perform the trigger action':

Flow is now waiting for a request to come in so it can show you how it executes through the path. Click 'Send' in Postman. You should be met with something like this back in Flow:

Awesome. It ran, checked the length, found it was > 1 and no-op'd as we expected. Let's set the First Name to null, set Flow to wait for another request (hit Edit, then Test again), and see what happens:

{
    "Name" : {
        "First" : null,
        "Last" : "Doe",
        "Title" : "Mr"
    },
    "Address" : {
        "Line1" : "1234 Anywhere St.",
        "Line2" : null,
        "City" : "Somewhere",
        "State" : "OR",
        "Zip" : "12345",
        "Country" : "United States of America"
    }
}

Ruh roh

Looks like we need to null check before length(). No problem. Click 'Edit' on your Flow, and click the '(+)' that appears above the Condition block when you hover between it and 'Initialize Variable'. Choose 'Add a condition':

This time, for our condition expression we need to check out First JSON property against null. We do this with the equals() expression, like so:

And now we can say if this is true, add an error message, otherwise move on to the other validation. To get 'true' you also must use the 'Expression' panel, just search for 'true'

So let's put our error message add in to the 'If yes' part just like we did the previous one:

but now we want our previous condition (length check) inside the 'false' condition for this check. Easy peasy, simply grab the icon for the length condition, and drag it in to the 'If no' space:

You should now have something that looks like this:

Let's give it another run with that same null input:

HUZZAH!

Now to inspect the outcome. If you click on the 'Append to array variable 2' step, you should see:

So we got a null First name, then we appended on to the array variable a new JSON object definition for the null failure. Sweet.

Now let's do a 1-character first name to watch it go:

{
    "Name" : {
        "First" : "J",
        "Last" : "Doe",
        "Title" : "Mr"
    },
    "Address" : {
        "Line1" : "1234 Anywhere St.",
        "Line2" : null,
        "City" : "Somewhere",
        "State" : "OR",
        "Zip" : "12345",
        "Country" : "United States of America"
    }
}

Perfect! That 2nd error message can now just say "less than 2 characters" but otherwise we're golden!

Now go through and implement the other validations we need.

TIP: You can rename any step in your Flow to be more readable, but only before another step takes a dependency on it. Try this to make your conditions more digestable instead of 'Condition X' all over the place 😉

You should end up with a Flow that looks something like this:

Now for the last part, returning the result to the caller.

Add a new action at the bottom, and search for 'Response'. Click 'See more' and choose 'Request':

Then choose 'Request - Response' and spit out our output variable:

Let's give 'er a run! Using our same request as before we should now get a response in Postman:

[
    {
        "id": 1,
        "message": "First name is null or less than 2 characters in length"
    }
]

PERFECT!

Tweak your request to trigger some/all of the other validations and watch the failures come back.

But, what if we want moar speed?? Let's do it all in parallel (aka Fan out/in pattern)!

Click the '+' under the 'Initialize variable' step and choose 'Add a condition'

Take condition #2 from your list, and drag it above the newly-created parallel condition:

Repeat for the other remaining conditions until you have a Flow that looks like this:

Now we just need to get the 'Response' action to show up after all the parallel executions have completed. Click 'New step' at the very bottom of the Flow and choose 'Add an action'. With the "Choose an action" dialog open, drag the existing 'Response' object to just below it:

This will create it outside the parallel executions. You can click 'Cancel' on the "Choose an action" dialog now.

Give our Flow an execution, and you'll see the validations are processed in parallel and can show up in our resulting array object in any order, depending on who completed execution first.

You can rename your Flow to something more intuitive by simply clicking its given name in the upper-left corner while editing

Hopefully you've found this an interesting foray in to the power of serverless computing and Microsoft's business-class serverless offering, Flow. It's certainly more powerful than it's given credit for (in my mind)!

At this point you're probably wondering "Ok so... what can't I use Flow for?" or "When shouldn't I use this tool, it's awesome!". Here's the thing, Flow is intended for business users to be used in a self-service manner. You can't do any of the more advanced business workflow operations like use an Integration Account to read schemas, etc with Flow. Also, as I showed earlier, Flow's fastest checks are at 1-minute intervals. This may not be adequate for higher-priority business operations. You only get the in-browser editor and as a result the ALM story is much more limited (test, prod only). From the administration side, Flow offers only DLP & licensing controls as well as O365 Sec & Compliance logs and DLP.

If you want to take Flow to the next level, stay tuned for next time when I show how to do the same process using Flow's IT Pro-grade offering: Logic Apps. More detail between these two can be found here.

I recommend sticking with Function access rights (vs Anonymous) for this one because it'd be an easy Function to abuse should anybody discover the URL. Once you've got that, here's the content to use to create two functions, one to convert JSON to XML and another to convert XML to JSON:

[FunctionName("ConvertToJson")]
public static IActionResult RunToJson([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]HttpRequest req, TraceWriter log)
{
    if (req.ContentType.IndexOf(@"/xml", 0, System.StringComparison.OrdinalIgnoreCase) == -1)
    {
        return new BadRequestObjectResult(@"Content-Type header must be an XML content type");
    }

    XmlDocument doc = new XmlDocument();
    doc.Load(req.Body);

    return new OkObjectResult(doc);
}

[FunctionName("ConvertToXml")]
public static async Task<HttpResponseMessage> RunToXmlAsync([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]HttpRequest req, TraceWriter log)
{
    if (req.ContentType.IndexOf(@"/json", 0, System.StringComparison.OrdinalIgnoreCase) == -1)
    {
        return new HttpResponseMessage(System.Net.HttpStatusCode.BadRequest)
        {
            Content = new StringContent(@"Content-Type header must be a JSON content type")
        };
    }

    var json = await req.ReadAsStringAsync();
    XmlDocument doc = JsonConvert.DeserializeXmlNode(json);

    StringBuilder output = new StringBuilder();
    using (var sw = new StringWriter(output))
        doc.WriteTo(new XmlTextWriter(sw));

    return new HttpResponseMessage(System.Net.HttpStatusCode.OK)
    {
        Content = new StringContent(output.ToString(), Encoding.Default, @"application/xml"),
    };
}

Dissecting ConvertToJson

As a set up for later, let's check out how simple it was to get XML -> JSON: Let's review some of the gymnastics and how I arrived at the XML -> JSON code shown above:

if (req.ContentType.IndexOf(@"/xml", 0, System.StringComparison.OrdinalIgnoreCase) == -1)
{
    return new BadRequestObjectResult(@"Content-Type header must be an XML content type");
}

XmlDocument doc = new XmlDocument();
doc.Load(req.Body);

return new OkObjectResult(doc);

Here was my test request in Postman:

POST /api/ConvertToJson HTTP/1.1
Host: localhost:7071
Content-Type: application/xml
Cache-Control: no-cache
Postman-Token: a5dc4ca4-b6dd-4193-b590-d15982219da7

<root>
	<this att="x">
		<underthis>val</underthis>
	</this>
	<that>
		<withval>x</withval>
		<bigval>
			<![CDATA[
			something something
			]]>
		</bigval>
	</that>
</root>

And here's what you get back:

Content-Type →application/json; charset=utf-8
Date →Fri, 25 May 2018 18:01:16 GMT
Server →Kestrel
Transfer-Encoding →chunked

{
    "root": {
        "this": {
            "@att": "x",
            "underthis": "val"
        },
        "that": {
            "withval": "x",
            "bigval": {
                "#cdata-section": "\n\t\t\tsomething something\n\t\t\t"
            }
        }
    }
}

Because Functions automatically takes any object given to OkObjectResult and runs it through JSON deserialization, simply giving it the XmlDocument resulting from LoadXml gives us exactly what we want! But this comes with some baggage...

Dissecting ConvertToXml

This one was even more bizarre.

if (req.ContentType.IndexOf(@"/json", 0, System.StringComparison.OrdinalIgnoreCase) == -1)
{
    return new HttpResponseMessage(System.Net.HttpStatusCode.BadRequest)
    {
        Content = new StringContent(@"Content-Type header must be a JSON content type")
    };
}

var json = await req.ReadAsStringAsync();
XmlDocument doc = JsonConvert.DeserializeXmlNode(json);

StringBuilder output = new StringBuilder();
using (var sw = new StringWriter(output))
    doc.WriteTo(new XmlTextWriter(sw));

return new HttpResponseMessage(System.Net.HttpStatusCode.OK)
{
    Content = new StringContent(output.ToString(), Encoding.Default, @"application/xml"),
};

The general approach to getting JSON in to XML is to take it and just serialize it to an XmlNode using Newtonsoft.Json's constructs. No biggie there, I did that. For starters, here's the Postman request we're going to be sending to ConvertToXml:

POST /api/ConvertToXml HTTP/1.1
Host: localhost:7071
Content-Type: application/json
Cache-Control: no-cache
Postman-Token: 7e55a73f-1d94-46b2-b93f-e7d1297c0c30

{
    "root": {
        "this": {
            "@att": "x",
            "underthis": "val"
        },
        "that": {
            "withval": "x",
            "bigval": {
                "#cdata-section": "\n\t\t\tsomething something\n\t\t\t"
            }
        }
    }
}

So now let's investigate why we can't just take the resulting XmlDocument object and write it out to the OkObjectResult. The first thing we have to change to experiment here is the return value of ConvertToXml. You'll notice it's set to HttpResponseMessage which is the type used, typically, in a v1 Azure Function, not a v2. More on that later, but change it back to IActionResult so the signature now looks like:

public static async Task<IActionResult> RunToXmlAsync([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]HttpRequest req, TraceWriter log)

Now, change the body to this:

if (req.ContentType.IndexOf(@"/json", 0, System.StringComparison.OrdinalIgnoreCase) == -1)
{
    return new BadRequestObjectResult(@"Content-Type header must be an JSON content type");
}

var json = await req.ReadAsStringAsync();
XmlDocument doc = JsonConvert.DeserializeXmlNode(json);

return new OkObjectResult(doc);

and give 'er a go, only to see JSON come out:

Content-Type →application/json; charset=utf-8
Date →Fri, 25 May 2018 17:44:59 GMT
Server →Kestrel
Transfer-Encoding →chunked

{
    "root": {
        "this": {
            "@att": "x",
            "underthis": "val"
        },
        "that": {
            "withval": "x",
            "bigval": {
                "#cdata-section": "\n\t\t\tsomething something\n\t\t\t"
            }
        }
    }
}

At this point, I tried adding [Produces(@"application/xml")] to the signature of my Function but it had no effect whatsoever. Functions must not respect those ASP.Net Core attributes, unfortunately (yet?). Let's try forcing the output to be XML using the MediaTypeFormatters collection again:

return new OkObjectResult(doc) { ContentTypes = new Microsoft.AspNetCore.Mvc.Formatters.MediaTypeCollection { @"application/xml" } };

Nada. This time we get HTTP 406 NOT ACCEPTABLE as the output of our Function. OK. Let's take the XML document, write it to a string, and spit that out.

XmlDocument doc = JsonConvert.DeserializeXmlNode(json);

StringBuilder sb = new StringBuilder();
using (var sw = new StringWriter(sb))
    doc.WriteTo(new XmlTextWriter(sw));

return new OkObjectResult(sb.ToString());

Gives us:

Content-Type →text/plain; charset=utf-8
Date →Fri, 25 May 2018 17:51:16 GMT
Server →Kestrel
Transfer-Encoding →chunked

<root><this att="x"><underthis>val</underthis></this><that><withval>x</withval><bigval><![CDATA[
			something something
			]]></bigval></that></root>

Close! But I really want that Content-Type header to be accurate, darnit! Let's add our MediaTypeFormatter back in:

return new OkObjectResult(sb.ToString()) { ContentTypes = new Microsoft.AspNetCore.Mvc.Formatters.MediaTypeCollection { @"application/xml" } };

Giving us.....

Content-Type →application/xml; charset=utf-8
Date →Fri, 25 May 2018 17:52:58 GMT
Server →Kestrel
Transfer-Encoding →chunked

<string xmlns="https://schemas.microsoft.com/2003/10/Serialization/">&lt;root&gt;&lt;this att="x"&gt;&lt;underthis&gt;val&lt;/underthis&gt;&lt;/this&gt;&lt;that&gt;&lt;withval&gt;x&lt;/withval&gt;&lt;bigval&gt;&lt;![CDATA[
			something something
			]]&gt;&lt;/bigval&gt;&lt;/that&gt;&lt;/root&gt;</string>

DAG NABBIT! (not even close to the actual words I was uttering at this point) Clearly ASP.Net Core and/or Functions is doing something automatic under the covers that I'm just not able to control. I knew ASP.Net MVC handles this kind of stuff beautifully; maybe .Net Core is just pushing us all to a JSON-only world? ¯_(ツ)_/¯ As a last-ditch effort I converted this Function to use the ASP.Net MVC constructs for response messages. Starting with the signature:

public static async Task<HttpResponseMessage> RunToXmlAsync([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]HttpRequest req, TraceWriter log)

Then each response I was sending back:

return new HttpResponseMessage(System.Net.HttpStatusCode.BadRequest)
{
    Content = new StringContent(@"Content-Type header must be a JSON content type")
};

return new HttpResponseMessage(System.Net.HttpStatusCode.OK)
{
    Content = new StringContent(output.ToString(), Encoding.Default, @"application/xml"),
};

and wouldn't you know...

Content-Length →149
Content-Type →application/xml; charset=utf-8
Date →Fri, 25 May 2018 17:57:08 GMT
Server →Kestrel

<root>
    <this att="x">
        <underthis>val</underthis>
    </this>
    <that>
        <withval>x</withval>
        <bigval>
            <![CDATA[
			something something
			]]>
        </bigval>
    </that>
</root>

TADA! We've got a body in XML, and we've got the header set to indicate it. Perfect! Having one function using the ASP.Net Core constructs (ConvertToJson) and one using ASP.Net MVC constructs doesn't seem to hurt matters at all, either. Enjoy! And hey, if you're an ASP.Net Core expert and see what I might be doing wrong in trying to get the desired headers & output for my Functions, please let me know in the comments!

Update

I posted my experience as a bug/feedback issue over on the Azure Functions Host repo and got a response from the (awesome) product team. They directed me to, instead, use one of two options:

• Accept header to trigger content negotiation between my Function and the Client (eg: send Accept: application/xml and the Function should return XML as both content-type header and the body payload)
• Use ContentResult instead of OkObjectResult and set the appropriate properties.

Let's walk through each of these.

For the Accept header, I changed my code for the XML part of the Function to look like this:

if (req.ContentType.IndexOf(@"/json", 0, System.StringComparison.OrdinalIgnoreCase) == -1)
{
    return new BadRequestObjectResult(@"Content-Type header must be a JSON content type");
}

var json = await req.ReadAsStringAsync();
XmlDocument doc = JsonConvert.DeserializeXmlNode(json);

return new OkObjectResult(doc);

Then my Postman requests becomes:

POST /api/ConvertToXml HTTP/1.1
Host: localhost:7071
Content-Type: application/json
Accept: application/xml
Cache-Control: no-cache
Postman-Token: a37ba05f-ae2a-4541-b44f-9d23edf44616

{
    "root": {
        "this": {
            "@att": "x",
            "underthis": "val"
        },
        "that": {
            "withval": "x",
            "bigval": {
                "#cdata-section": "\n\t\t\tsomething something\n\t\t\t"
            }
        }
    }
}

But unfortunately I didn't see what I expected:

{
    "root": {
        "this": {
            "@att": "x",
            "underthis": "val"
        },
        "that": {
            "withval": "x",
            "bigval": {
                "#cdata-section": "\n\t\t\tsomething something\n\t\t\t"
            }
        }
    }
}

If I change the function to, instead of returning the XmlDocument object to returning the string we built up previously:

XmlDocument doc = JsonConvert.DeserializeXmlNode(json);

StringBuilder output = new StringBuilder();
using (var sw = new StringWriter(output))
    doc.WriteTo(new XmlTextWriter(sw));

return new OkObjectResult(output.ToString());

I get back:

<string xmlns="https://schemas.microsoft.com/2003/10/Serialization/">&lt;root&gt;&lt;this att="x"&gt;&lt;underthis&gt;val&lt;/underthis&gt;&lt;/this&gt;&lt;that&gt;&lt;withval&gt;x&lt;/withval&gt;&lt;bigval&gt;&lt;![CDATA[
			something something
			]]&gt;&lt;/bigval&gt;&lt;/that&gt;&lt;/root&gt;</string>

which looks familiar, but is still incorrect.

Let's check out this ContentResult response. Our code now becomes:

XmlDocument doc = JsonConvert.DeserializeXmlNode(json);

StringBuilder output = new StringBuilder();
using (var sw = new StringWriter(output))
    doc.WriteTo(new XmlTextWriter(sw));

return new ContentResult
{
    Content = output.ToString(),
    ContentType = @"application/xml",
    StatusCode = StatusCodes.Status200OK
};

and we get back:

<root>
    <this att="x">
        <underthis>val</underthis>
    </this>
    <that>
        <withval>x</withval>
        <bigval>
            <![CDATA[
			something something
			]]>
        </bigval>
    </that>
</root>

Yay! We're now fully within the .Net Core arena and returning back XML. It's also worth noting that with the ContentResult approach the Accept header on the request is inconsequential.

As you know, I wrote a blog post a while back about creating a Microsoft Flow integration that starts up a Minecraft server in Azure for my kids. After a few months of using this setup, I became annoyed by how many times the kids had to send me a message, call me, etc just to ask me to start the server for them. After being involved in our Bot Framework space for a while and now focusing on our Serverless platform the thought occurred to me - why not just make this a bot they can ask to start the server??

By making my MS Flow accessible via an HTTP request, this becomes quite easy. Simply architect your conversation, ship it to a place it can be hit from Bot Framework (Azure App Service, etc), and you're done!

But what if we wanted to take it a step further, what if we wanted to quickly & easily ship an Alexa Skill that turns on a server in Azure? Turns out with Azure Functions and a great Alexa .Net package from my colleague Tim Heuer this is incredibly quick and easy!

Create your Azure Function

Fire up Visual Studio 2017, and create a new Azure Function

For Alexa integration, your Azure Function needs to be an HTTP Endpoint (a v2 Function works great since it's based on .Net Standard):

After your project is set up, you need to add only one additional nuget package to it:

Install-Package Alexa.NET

Implement Alexa's handshake protocol

Now we're ready to integrate with Alexa. If you've ever of or done Alexa Skill development, you know there's a pretty rigorous authentication path that an Alexa Skill takes to validate that your endpoint is good to communicate with the Alexa Skill backend and is the endpoint that's supposed to be handling the requests. Fortunately, Alexa.NET makes this easy as pie to handle!

Modify the signature of your function method to accept Function-authorized POST requests (it's easy to embed the query string parameter in to your Alexa Skill's configuration):

public static async Task<IActionResult> RunAsync([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]HttpRequest req, TraceWriter log)

Next, clear out the content of your Function's handler, and replace it with this:

if (!(await IsValid(req)))
{
    return new BadRequestResult();
}

return new OkResult();

and add an IsValid method in to your Function class that looks like this:

private static async Task<bool> IsValid(HttpRequest request, TraceWriter log)
{
    // Verify SignatureCertChainUrl is present
    request.Headers.TryGetValue("SignatureCertChainUrl", out var signatureChainUrl);
    if (String.IsNullOrWhiteSpace(signatureChainUrl))
    {
        log.Error(@"Alexa - empty Signature Cert Chain Url header");
        return false;
    }

    Uri certUrl;
    try
    {
        certUrl = new Uri(signatureChainUrl);
    }
    catch
    {
        log.Error($@"Alexa - Unable to put sig chain url in to Uri: {signatureChainUrl}");
        return false;
    }

    // Verify SignatureCertChainUrl is Signature
    request.Headers.TryGetValue("Signature", out var signature);
    if (String.IsNullOrWhiteSpace(signature))
    {
        log.Error(@"Alexa - empty Signature header");
        return false;
    }

    // rewind body so we get the full payload
    request.Body.Position = 0;
    var body = await request.ReadAsStringAsync();

    // rewind so caller gets body back at the beginning
    request.Body.Position = 0;

    if (String.IsNullOrWhiteSpace(body))
    {
        log.Error(@"Alexa - empty Body");
        return false;
    }

    var valid = await RequestVerification.Verify(signature, certUrl, body);

    if (!valid)
    {
        log.Error(@"Alexa - RequestVerification.Verify failed");
    }
    return valid;
}

With this done, your Skill will now properly respond to requests from Alexa and validate them when appropriate (eg: whether in testing, beta, or production).

Write the conversation

Now let's get to the meat of implementing your conversation.

There was one interesting thing I learned while I was developing my first Alexa Skill which I didn't anticipate: a skill can never take more than 8 seconds to end its turn, period. While you can add "progress" responses to what Alexa sends back, these don't extend the amount of time you get to "complete" the ask from the user.

For me this meant I needed to rethink how I was executing my task. Initially, I wanted Alexa to tell my kids when the server was finally up and running, or when the shutdown was complete. The Azure Automation task in a Logic App has the "Wait for Finish" option but of course this task almost never completes in under 8 seconds. This meant I had to change how I executed my Start/Stop asks to instead use the Azure Management REST API; a much quicker approach. But I digress.

Let's see how we write conversation in an Alexa Skill using Alexa.NET.

The first thing to do is parse the payload that came in to your skill once you've determined it's all good. To do this, simply take the JSON that was posted and turn it in to an SkillRequest object from Alexa.NET like so:

var requestPayload = await req.ReadAsStringAsync();
var alexaRequest = JsonConvert.DeserializeObject<SkillRequest>(requestPayload);

Much like a Cortana Skill, Alexa kicks off your skill with an "Intent" to which your code reacts. Alexa.NET nicely maps these to strong types so you get to do the following:

SkillResponse response = null;
try
{
    if (alexaRequest.Request is LaunchRequest launch)
    {
        response = ResponseBuilder.Ask($@"Welcome to the Minecraft bot. Whatchya wanna do.",
            new Reprompt { OutputSpeech = new PlainTextOutputSpeech { Text = @"Sorry, I don't understand. You can say start the server, stop the server, or is it on" } });
    }
    else if (alexaRequest.Request is IntentRequest intent)

Here you see me handling if my kids say "Alexa, Start Minecraft Bot". This will come through from the backend as a LaunchRequest intent which I can handle.

This also introduces us to the one-turn nature of Alexa. Every request to your skill must be met with, in Alexa.NET, a single SkillResponse result which must be returned at the end of the handler. For me the code became easiest if I set a single response variable, and at the bottom of my Function's code simply return response; (you'll see this in a bit with the finished product).

Also much like Cortana, Alexa maps what your users say, outside "launch", "open", "start" and the like, to intents you define in the Developer Portal (more on this later). For me, the flows I anticipated handling were to start the server, stop it, or check to see if it's on. So this code goes on to look like so:

if (intent.Intent.Name.Equals(@"is_it_on"))
{
...
}
else if (intent.Intent.Name.Equals(@"start_server"))
{
...
}
else if (intent.Intent.Name.Equals(@"stop_server"))
{
...
}
else
{
...
}

In each handler for the intent, then, we simply implement the work to do and the response to send back. Here's what it looks like for is_it_on:

var prog = new ProgressiveResponse(alexaRequest);
if (prog.CanSend())
{
    await prog.SendSpeech(@"One second, checking.");
}

if (IsVmStarted(_config))
{
    response = ResponseBuilder.Tell(@"Yup, it's up and running!");
}
else
{
    response = ResponseBuilder.Ask(@"Nope it's off. Would you like to start it?",
        new Reprompt { OutputSpeech = new PlainTextOutputSpeech { Text = @"Sorry, I don't understand. Just say yes or no" } });
}

Here we are also introduced to the concept of a "Progressive Response". Now, remember, even though you've sent a response back as your Skill is running, you still only get 8 seconds to set & return the response object we're populating. You can see the Progressive Response is sent to the user right away by the fact you have to await the SendSpeech method.

This also introduces the concept of an 'Ask' from Alexa. For my skill, if the server isn't on I let the kids know and give them the option of starting it right away with this prompt. We'll go in to more of how this gets defined in the Skill portal soon.

There are two other intents for which I also implemented handling: start_server where I tell the kids "ok I've sent the request!" and stop_server where I simply issue the REST request and let them know it's shutting down.

Handling the answer to a question, aka an "Ask", from Alexa

I mentioned previously that when the kids ask for the status of the server, I go ahead and let them know that it's on/off and, if it's off, I ask if they want me to turn it on. If they answer 'yes' here, obviously I want to run that routine & let them know it's turning on. But how do I know that they're response ("yes", "no") was in reply to Alexa's ask, and not something else? Enter the 'prompt' value in each request coming in from Alexa.

In the portal, you can configure prompts for any intent (ie: a prompt that comes back from Alexa after it's handled a particular intent). To do this, you have a fill out a couple of areas in the Alexa Skills Kit console:

1. Define a slot type for expected responses from your user to your prompt.

Here you can see I've created a new slot type called PROMPTOPTIONS. I'll use this later. Here's what it looks like:

Take special note of the value and synonyms areas. The value is what you'll be checking for in code, the synonyms are used to train the underlying Natural Language (NL) model for each of the values so, in theory, my kids should be able to say "yah" and it should get mapped to yes even though I haven't explictly defined that as a synonym.

Once you've defined your slot type, it's time to tell the is_it_on intent to use it now

2. Direct an existing intent to use the new slot type

In my is_it_on intent I now add an Intent Slot, call it prompt and map it to the new PROMPTOPTIONS slot type. Additionally, I add an utterance for my intent that says the user can say one of the slot values (eg: "yes", "no") and it should trigger this intent. This seems a little problematic in my mind; I feel like my kids could say "start minecraft bot" to Alexa, and when she answers back "Whatchya wanna do?" they could just say "yes" and end up triggering the is_it_on intent which wouldn't be desirable; I haven't tried this flow yet, though.

After making these two changes, click 'Save Model' and 'Build Model' to make the new language model active for your skill.

Now let's head back to the code:

response = ResponseBuilder.Ask(@"Nope it's off. Would you like to start it?",
    new Reprompt { OutputSpeech = new PlainTextOutputSpeech { Text = @"Sorry, I don't understand. Just say yes or no" } });

Here we tell Alexa to ask the user a question. The user's next response will come back in under the same Intent as it was asked during (eg: is_it_on) but with slots filled in accordingly. So the handling code for my is_it_on intent now looks like this:

if (intent.Intent.Name.Equals(@"is_it_on"))
{
    string promptResponse = intent.Intent.Slots[@"prompt"]?.Value;
    if (!string.IsNullOrEmpty(promptResponse))
    {
        if (promptResponse.Equals("yes", StringComparison.OrdinalIgnoreCase))
        {
            response = StartServer(alexaRequest);
        }
        else
        {
            response = ResponseBuilder.Empty();
        }
    }
    else
    {
        _logger.TrackEvent(@"AlexaIsItOn");
        var prog = new ProgressiveResponse(alexaRequest);
        if (prog.CanSend())
        {
            await prog.SendSpeech(@"One second, checking.");
...

Where you can now see I'm checking that slot we created, prompt, to see if it's been populated with anything and, if it has and it's the yes value, go ahead and turn on the server. Otherwise, just stop responding (ResponseBuilder.Empty()).

Sparing the full detail of the bot (calling in to Azure Management API to do the work, etc), hopefully you can now see how quick & easy it is to write an Alexa Skill using Azure Functions and Alexa.Net! Curious how much this may end up costing you? I think you might be quite surprised.

To make this easier, here's a class that'll make this a one-line operation in your code:

Update

Since posting this a reader has brought to my attention issues with using HttpClient as a disposable object. Thank you!!

The implementation, therefore, might be a bit different than what was originally posted here in Nov 2017. But it's all for the better I promise! 😃

class EventGridObject
{
    private static readonly HttpClient _httpClient = new HttpClient();

    private readonly Uri _endpoint;
    private readonly string _sasKey;

    public EventGridObject(string topicEndpoint, string sasKey, string subject, string eventType, string id = null, DateTime? eventTime = null) : this(new Uri(topicEndpoint), sasKey, subject, eventType, id, eventTime) { }
    public EventGridObject(Uri topicEndpoint, string sasKey, string subject, string eventType, string id = null, DateTime? eventTime = null)
    {
        _endpoint = topicEndpoint ?? throw new ArgumentNullException(nameof(topicEndpoint));
        _sasKey = !string.IsNullOrWhiteSpace(sasKey) ? sasKey : throw new ArgumentNullException(nameof(sasKey));
        Id = string.IsNullOrWhiteSpace(id) ? Guid.NewGuid().ToString() : id;
        Subject = !string.IsNullOrWhiteSpace(subject) ? subject : throw new ArgumentNullException(nameof(subject));
        EventType = !string.IsNullOrWhiteSpace(eventType) ? eventType : throw new ArgumentNullException(nameof(eventType));
        EventTime = eventTime ?? DateTime.UtcNow;

        // https://byterot.blogspot.co.uk/2016/07/singleton-httpclient-dns.html
        System.Net.ServicePointManager.FindServicePoint(_endpoint).ConnectionLeaseTimeout = (int)TimeSpan.FromMinutes(1).TotalMilliseconds;
    }

    [JsonProperty(@"data")]
    public object Data { get; set; }
    [JsonProperty(@"id")]
    public string Id { get; }
    [JsonProperty(@"subject")]
    public string Subject { get; }
    [JsonProperty(@"eventType")]
    public string EventType { get; }
    [JsonProperty(@"eventTime")]
    public DateTime EventTime { get; }

    public async Task<HttpResponseMessage> SendAsync()
    {
        if (_httpClient.DefaultRequestHeaders.Contains(@"aeg-sas-key"))
            _httpClient.DefaultRequestHeaders.Remove(@"aeg-sas-key");
        _httpClient.DefaultRequestHeaders.Add(@"aeg-sas-key", _sasKey);

        return await _httpClient.PostAsJsonAsync(_endpoint, new[] { this });
    }
}

and you use it like this:

await new EventGridObject(@"https://<grid topic>.westus2-1.eventgrid.azure.net/api/events", @"Yo58SQ...=", @"my/subject/data/goes/here", @"myEventType")
{
    Data = new
    {
        myproperty1 = @"value1",
        myprop2 = variable2,
        queuedTime = DateTime.UtcNow
    }
}.SendAsync();

1. Make sure you run ngrok correctly. When launching ngrok, you have the option of specifying the port it forwards to locally. For .Net Azure Functions this is, by default, 7071 so you need to give that to ngrok when you run it. Second, you need to specify the host header as Azure Functions runtime doesn't like seeing the request came in to 'ngrok.io' but is now hitting its endpoint hosted on 'localhost'. To do this, you specify the host-header parameter. All-in-all, your ngrok launch command should look something like this (for Azure Functions .Net development): ngrok http -host-header=localhost 7071

2. Your Function needs to properly handle the SubscriptionValidationEvent sent out by Event Grid when a subscription endpoint is created. You can read more about this here, but suffice to say here's the code you need to do this:

1. ngrok endpoints change with each run of ngrok. The nice thing is if you leave ngrok running and hibernate/suspend your machine then wake it back up, ngrok will attempt to reconnect and grant you the same address. Very handy. However if the address changes, or you stop/restart ngrok, don't forget to update your Event Grid Subscription's endpoint URL!
2. Don't forget to have your function running locally when you create the subscription! If Event Grid doesn't get a 200 OK back with the right payload, it'll error out creating the subscription for you and you'll have to do it again.

https://prod-14.westcentralus.logic.azure.com:443/workflows/.../triggers/manual/paths/invoke?api-version...

1. Tightly couples our app and our backend endpoint; this limits our ability to change to a new endpoint, service, provider, etc later without impacting clients
2. Exposes the URL to the outside world (javascript, etc) which...
3. Makes the URL (more) vulnerable to a Denial of Wallet attack

Display Name: My API
Name: endpoint
Description: Here is where a description for My API lives
Web service URL: https://requestb.in/1kzvq4a1
URL Scheme: Both
API URL suffix: safe-endpoint
Version? Yes
Products: Unlimited
Versioning scheme: Query string
Version Identifier: v1
Verison Parameter: api

1. Click My API's v1 instance
2. Click the 'edit' pencil on the Inbound Processing area; this is where we'll add a policy regarding the rate at which we can call our API.
3. Position your cursor below the <base /> tag under the <inbound> tag
4. Click "Limit call rate per key"

1. Click 'Add operation'
2. Choose POST and enter / for the resource path
3. Display Name, let's just call it post

Asp.Net Core Middleware

Custom Middleware

How Middleware works

Writing our middleware

1. A new exception type - HttpStatusCodeException - which I would throw whenever I wanted to stop execution of the current request and send back a response with a specific Status Code and Body. It turned out looking like this:

2. The middleware handler of the new exception. This middleware would need to trap the exception, set the necessary properties on the response, send it back and, most importantly, not let any more middleware fire off. It ended up looking like this:

Using our middleware

The result