Processing Azure Data Factory Event Trigger Properties

Question

I have a data factory which triggers based on storage blob event. In the triggered event, I see two properties TriggerTime and EventPayload. As I have need to read the Storage Blob related information I am trying to process the EventPayload in the Data Factory. I would like access a property like 'url' from the data tag.

A sample payload looks like this:

{
   "topic":"/subscriptions/7xxxxe5bbccccc85/resourceGroups/das00/providers/Microsoft.Storage/storageAccounts/datxxxxxx61",
   "subject":"/blobServices/default/containers/raw/blobs/sample.parquet",
   "eventType":"Microsoft.Storage.BlobCreated",
   "id":"a1c320d7-501f-0047-362c-xxxxxxxxxxxx",
   "data":{
      "api":"FlushWithClose",
      "requestId":"5010",
      "eTag":"0x8D82743B5D86E72",
      "contentType":"application/octet-stream",
      "contentLength":203665463,
      "contentOffset":0,
      "blobType":"BlockBlob",
      "url":"https://mystorage.dfs.core.windows.net/raw/sample.parquet",
      "sequencer":"000000000000000000000000000066f10000000000000232",
      "storageDiagnostics":{
         "batchId":"89308627-6e28-xxxxx-96e2-xxxxxx"
      }
   },
   "dataVersion":"3",
   "metadataVersion":"1",
   "eventTime":"2020-07-13T15:45:04.0076557Z"
}

Is there any short hand for processing the EventPayload in the Data Factory? For example, the filename and folderpath of an event can be accessed using @triggerBody() in the Data Factory. Does this require custom code like Azure function?

Answer 1

Hello @SatyaD-5902 and thank you for your question. Your ask can be accomplished without using Azure Function.

In the document how-to-create-event-trigger, an example of how to fetch some properties of the trigger is given by

@trigger().outputs.body.folderPath

This is only a subset of what is available. I can see all of what is available by using

@{trigger{}}

Nested inside the trigger, I find the payload you refer to.

Below I show cleaned up output of using @{trigger()}.

{
    "name": "Trigger...",
    "outputs": {
        "headers": {
            "Connection": "Keep-Alive",
            ...
            "Content-Type": "application/json; charset=utf-8"
        },
        "body": {
            "event": {
                "topic":  ...

To make things easier to access, change the parameter type to object. Then when setting up the trigger parameter, pass the section you want to work on like:

@trigger().outputs.body.event)

Then you can take the parameter, and use it in a set variable to get the individual properties, such as the url you mentioned:

@pipeline().parameters.triggerStuff.data.url