Dataflow - Create Or Update

Reference

Service:: IoT Operations

API Version:: 2024-08-15-preview

Create a DataflowResource

PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.IoTOperations/instances/{instanceName}/dataflowProfiles/{dataflowProfileName}/dataflows/{dataflowName}?api-version=2024-08-15-preview

URI Parameters

Name	In	Required	Type	Description
dataflowName	path	True	string	Name of Instance dataflowProfile dataflow resource Regex pattern: `^[a-z0-9][a-z0-9-]*[a-z0-9]$`
dataflowProfileName	path	True	string	Name of Instance dataflowProfile resource Regex pattern: `^[a-z0-9][a-z0-9-]*[a-z0-9]$`
instanceName	path	True	string	Name of instance. Regex pattern: `^[a-z0-9][a-z0-9-]*[a-z0-9]$`
resourceGroupName	path	True	string	The name of the resource group. The name is case insensitive.
subscriptionId	path	True	string uuid	The ID of the target subscription. The value must be an UUID.
api-version	query	True	string	The API version to use for this operation.

Request Body

Name	Required	Type	Description
extendedLocation	True	ExtendedLocation	Edge location of the resource.
properties		DataflowProperties	The resource-specific properties for this resource.

Responses

Name	Type	Description
200 OK	DataflowResource	Resource 'DataflowResource' update operation succeeded
201 Created	DataflowResource	Resource 'DataflowResource' create operation succeeded Headers Azure-AsyncOperation: string Retry-After: integer
Other Status Codes	ErrorResponse	An unexpected error response.

Name

Type

Description

200 OK

DataflowResource

Resource 'DataflowResource' update operation succeeded

201 Created

DataflowResource

Resource 'DataflowResource' create operation succeeded

Headers

Azure-AsyncOperation: string
Retry-After: integer

Other Status Codes

ErrorResponse

An unexpected error response.

Security

azure_auth

Azure Active Directory OAuth2 Flow.

Type: oauth2
Flow: implicit
Authorization URL: https://login.microsoftonline.com/common/oauth2/authorize

Scopes

Name	Description
user_impersonation	impersonate your user account

Examples

Dataflow_CreateOrUpdate

Dataflow_CreateOrUpdate_ComplexContextualization

Dataflow_CreateOrUpdate_ComplexEventHub

Dataflow_CreateOrUpdate_FilterToTopic

Dataflow_CreateOrUpdate_SimpleEventGrid

Dataflow_CreateOrUpdate_SimpleFabric

Dataflow_CreateOrUpdate

Sample request

HTTP

PUT https://management.azure.com/subscriptions/F8C729F9-DF9C-4743-848F-96EE433D8E53/resourceGroups/rgiotoperations/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123?api-version=2024-08-15-preview

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "knnafvkwoeakm",
        "sourceSettings": {
          "endpointRef": "iixotodhvhkkfcfyrkoveslqig",
          "assetRef": "zayyykwmckaocywdkohmu",
          "serializationFormat": "Json",
          "schemaRef": "pknmdzqll",
          "dataSources": [
            "chkkpymxhp"
          ]
        },
        "builtInTransformationSettings": {
          "serializationFormat": "Delta",
          "schemaRef": "mcdc",
          "datasets": [
            {
              "key": "qsfqcgxaxnhfumrsdsokwyv",
              "description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
              "schemaRef": "n",
              "inputs": [
                "mosffpsslifkq"
              ],
              "expression": "aatbwomvflemsxialv"
            }
          ],
          "filter": [
            {
              "type": "Filter",
              "description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
              "inputs": [
                "sxmjkbntgb"
              ],
              "expression": "n"
            }
          ],
          "map": [
            {
              "type": "NewProperties",
              "description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
              "inputs": [
                "xsbxuk"
              ],
              "expression": "txoiltogsarwkzalsphvlmt",
              "output": "nvgtmkfl"
            }
          ]
        },
        "destinationSettings": {
          "endpointRef": "kybkchnzimerguekuvqlqiqdvvrt",
          "dataDestination": "cbrh"
        }
      }
    ]
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  }
}

Sample response

Status code:: 200

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "knnafvkwoeakm",
        "sourceSettings": {
          "endpointRef": "iixotodhvhkkfcfyrkoveslqig",
          "assetRef": "zayyykwmckaocywdkohmu",
          "serializationFormat": "Json",
          "schemaRef": "pknmdzqll",
          "dataSources": [
            "chkkpymxhp"
          ]
        },
        "builtInTransformationSettings": {
          "serializationFormat": "Delta",
          "schemaRef": "mcdc",
          "datasets": [
            {
              "key": "qsfqcgxaxnhfumrsdsokwyv",
              "description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
              "schemaRef": "n",
              "inputs": [
                "mosffpsslifkq"
              ],
              "expression": "aatbwomvflemsxialv"
            }
          ],
          "filter": [
            {
              "type": "Filter",
              "description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
              "inputs": [
                "sxmjkbntgb"
              ],
              "expression": "n"
            }
          ],
          "map": [
            {
              "type": "NewProperties",
              "description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
              "inputs": [
                "xsbxuk"
              ],
              "expression": "txoiltogsarwkzalsphvlmt",
              "output": "nvgtmkfl"
            }
          ]
        },
        "destinationSettings": {
          "endpointRef": "kybkchnzimerguekuvqlqiqdvvrt",
          "dataDestination": "cbrh"
        }
      }
    ],
    "provisioningState": "Succeeded"
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  },
  "id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
  "name": "jxhcpwgfkxqasbexkookvxk",
  "type": "zkuozvgjseokfchkscoswthzjdry",
  "systemData": {
    "createdBy": "ssvaslsmudloholronopqyxjcu",
    "createdByType": "User",
    "createdAt": "2024-08-09T18:13:29.389Z",
    "lastModifiedBy": "gnicpuszwd",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2024-08-09T18:13:29.389Z"
  }
}

Status code:: 201

Azure-AsyncOperation: https://contoso.com/operationstatus

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "knnafvkwoeakm",
        "sourceSettings": {
          "endpointRef": "iixotodhvhkkfcfyrkoveslqig",
          "assetRef": "zayyykwmckaocywdkohmu",
          "serializationFormat": "Json",
          "schemaRef": "pknmdzqll",
          "dataSources": [
            "chkkpymxhp"
          ]
        },
        "builtInTransformationSettings": {
          "serializationFormat": "Delta",
          "schemaRef": "mcdc",
          "datasets": [
            {
              "key": "qsfqcgxaxnhfumrsdsokwyv",
              "description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
              "schemaRef": "n",
              "inputs": [
                "mosffpsslifkq"
              ],
              "expression": "aatbwomvflemsxialv"
            }
          ],
          "filter": [
            {
              "type": "Filter",
              "description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
              "inputs": [
                "sxmjkbntgb"
              ],
              "expression": "n"
            }
          ],
          "map": [
            {
              "type": "NewProperties",
              "description": "Lorem ipsum odor amet, consectetuer adipiscing elit.",
              "inputs": [
                "xsbxuk"
              ],
              "expression": "txoiltogsarwkzalsphvlmt",
              "output": "nvgtmkfl"
            }
          ]
        },
        "destinationSettings": {
          "endpointRef": "kybkchnzimerguekuvqlqiqdvvrt",
          "dataDestination": "cbrh"
        }
      }
    ],
    "provisioningState": "Succeeded"
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  },
  "id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
  "name": "jxhcpwgfkxqasbexkookvxk",
  "type": "zkuozvgjseokfchkscoswthzjdry",
  "systemData": {
    "createdBy": "ssvaslsmudloholronopqyxjcu",
    "createdByType": "User",
    "createdAt": "2024-08-09T18:13:29.389Z",
    "lastModifiedBy": "gnicpuszwd",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2024-08-09T18:13:29.389Z"
  }
}

Dataflow_CreateOrUpdate_ComplexContextualization

Sample request

HTTP

PUT https://management.azure.com/subscriptions/F8C729F9-DF9C-4743-848F-96EE433D8E53/resourceGroups/rgiotoperations/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/aio-to-adx-contexualized?api-version=2024-08-15-preview

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "azure-iot-operations/data/thermostat"
          ]
        }
      },
      {
        "operationType": "BuiltInTransformation",
        "name": "transformation1",
        "builtInTransformationSettings": {
          "map": [
            {
              "inputs": [
                "*"
              ],
              "output": "*"
            },
            {
              "inputs": [
                "$context(quality).*"
              ],
              "output": "enriched.*"
            }
          ],
          "datasets": [
            {
              "key": "quality",
              "inputs": [
                "$source.country",
                "$context.country"
              ],
              "expression": "$1 == $2"
            }
          ]
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "adx-endpoint",
          "dataDestination": "mytable"
        }
      }
    ]
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  }
}

Sample response

Status code:: 200

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "azure-iot-operations/data/thermostat"
          ]
        }
      },
      {
        "operationType": "BuiltInTransformation",
        "name": "transformation1",
        "builtInTransformationSettings": {
          "map": [
            {
              "inputs": [
                "*"
              ],
              "output": "*"
            },
            {
              "inputs": [
                "$context(quality).*"
              ],
              "output": "enriched.*"
            }
          ],
          "datasets": [
            {
              "key": "quality",
              "inputs": [
                "$source.country",
                "$context.country"
              ],
              "expression": "$1 == $2"
            }
          ]
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "adx-endpoint",
          "dataDestination": "mytable"
        }
      }
    ],
    "provisioningState": "Succeeded"
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  },
  "id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
  "name": "jxhcpwgfkxqasbexkookvxk",
  "type": "zkuozvgjseokfchkscoswthzjdry",
  "systemData": {
    "createdBy": "ssvaslsmudloholronopqyxjcu",
    "createdByType": "User",
    "createdAt": "2024-08-09T18:13:29.389Z",
    "lastModifiedBy": "gnicpuszwd",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2024-08-09T18:13:29.389Z"
  }
}

Status code:: 201

Azure-AsyncOperation: https://contoso.com/operationstatus

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "azure-iot-operations/data/thermostat"
          ]
        }
      },
      {
        "operationType": "BuiltInTransformation",
        "name": "transformation1",
        "builtInTransformationSettings": {
          "map": [
            {
              "inputs": [
                "*"
              ],
              "output": "*"
            },
            {
              "inputs": [
                "$context(quality).*"
              ],
              "output": "enriched.*"
            }
          ],
          "datasets": [
            {
              "key": "quality",
              "inputs": [
                "$source.country",
                "$context.country"
              ],
              "expression": "$1 == $2"
            }
          ]
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "adx-endpoint",
          "dataDestination": "mytable"
        }
      }
    ],
    "provisioningState": "Succeeded"
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  },
  "id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
  "name": "jxhcpwgfkxqasbexkookvxk",
  "type": "zkuozvgjseokfchkscoswthzjdry",
  "systemData": {
    "createdBy": "ssvaslsmudloholronopqyxjcu",
    "createdByType": "User",
    "createdAt": "2024-08-09T18:13:29.389Z",
    "lastModifiedBy": "gnicpuszwd",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2024-08-09T18:13:29.389Z"
  }
}

Dataflow_CreateOrUpdate_ComplexEventHub

Sample request

HTTP

PUT https://management.azure.com/subscriptions/F8C729F9-DF9C-4743-848F-96EE433D8E53/resourceGroups/rgiotoperations/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/aio-to-event-hub-transformed?api-version=2024-08-15-preview

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "azure-iot-operations/data/thermostat"
          ]
        }
      },
      {
        "operationType": "BuiltInTransformation",
        "builtInTransformationSettings": {
          "filter": [
            {
              "inputs": [
                "temperature.Value",
                "\"Tag 10\".Value"
              ],
              "expression": "$1 > 9000 && $2 >= 8000"
            }
          ],
          "map": [
            {
              "inputs": [
                "*"
              ],
              "output": "*"
            },
            {
              "inputs": [
                "temperature.Value",
                "\"Tag 10\".Value"
              ],
              "expression": "($1+$2)/2",
              "output": "AvgTemp.Value"
            },
            {
              "inputs": [],
              "expression": "true",
              "output": "dataflow-processed"
            },
            {
              "inputs": [
                "temperature.SourceTimestamp"
              ],
              "expression": "",
              "output": ""
            },
            {
              "inputs": [
                "\"Tag 10\""
              ],
              "expression": "",
              "output": "pressure"
            },
            {
              "inputs": [
                "temperature.Value"
              ],
              "expression": "cToF($1)",
              "output": "temperatureF.Value"
            },
            {
              "inputs": [
                "\"Tag 10\".Value"
              ],
              "expression": "scale ($1,0,10,0,100)",
              "output": "\"Scale Tag 10\".Value"
            }
          ]
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "event-hub-endpoint",
          "dataDestination": "myuniqueeventhub"
        }
      }
    ]
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  }
}

Sample response

Status code:: 200

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "azure-iot-operations/data/thermostat"
          ]
        }
      },
      {
        "operationType": "BuiltInTransformation",
        "name": "transformation1",
        "builtInTransformationSettings": {
          "map": [
            {
              "inputs": [
                "*"
              ],
              "output": "*"
            },
            {
              "inputs": [
                "$context(quality).*"
              ],
              "output": "enriched.*"
            }
          ],
          "datasets": [
            {
              "key": "quality",
              "inputs": [
                "$source.country",
                "$context.country"
              ],
              "expression": "$1 == $2"
            }
          ]
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "adx-endpoint",
          "dataDestination": "mytable"
        }
      }
    ],
    "provisioningState": "Succeeded"
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  },
  "id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
  "name": "jxhcpwgfkxqasbexkookvxk",
  "type": "zkuozvgjseokfchkscoswthzjdry",
  "systemData": {
    "createdBy": "ssvaslsmudloholronopqyxjcu",
    "createdByType": "User",
    "createdAt": "2024-08-09T18:13:29.389Z",
    "lastModifiedBy": "gnicpuszwd",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2024-08-09T18:13:29.389Z"
  }
}

Status code:: 201

Azure-AsyncOperation: https://contoso.com/operationstatus

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "azure-iot-operations/data/thermostat"
          ]
        }
      },
      {
        "operationType": "BuiltInTransformation",
        "name": "transformation1",
        "builtInTransformationSettings": {
          "map": [
            {
              "inputs": [
                "*"
              ],
              "output": "*"
            },
            {
              "inputs": [
                "$context(quality).*"
              ],
              "output": "enriched.*"
            }
          ],
          "datasets": [
            {
              "key": "quality",
              "inputs": [
                "$source.country",
                "$context.country"
              ],
              "expression": "$1 == $2"
            }
          ]
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "adx-endpoint",
          "dataDestination": "mytable"
        }
      }
    ],
    "provisioningState": "Succeeded"
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  },
  "id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
  "name": "jxhcpwgfkxqasbexkookvxk",
  "type": "zkuozvgjseokfchkscoswthzjdry",
  "systemData": {
    "createdBy": "ssvaslsmudloholronopqyxjcu",
    "createdByType": "User",
    "createdAt": "2024-08-09T18:13:29.389Z",
    "lastModifiedBy": "gnicpuszwd",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2024-08-09T18:13:29.389Z"
  }
}

Dataflow_CreateOrUpdate_FilterToTopic

Sample request

HTTP

PUT https://management.azure.com/subscriptions/F8C729F9-DF9C-4743-848F-96EE433D8E53/resourceGroups/rgiotoperations/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/mqtt-filter-to-topic?api-version=2024-08-15-preview

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "azure-iot-operations/data/thermostat"
          ]
        }
      },
      {
        "operationType": "BuiltInTransformation",
        "name": "transformation1",
        "builtInTransformationSettings": {
          "filter": [
            {
              "type": "Filter",
              "description": "filter-datapoint",
              "inputs": [
                "temperature.Value",
                "\"Tag 10\".Value"
              ],
              "expression": "$1 > 9000 && $2 >= 8000"
            }
          ],
          "map": [
            {
              "type": "PassThrough",
              "inputs": [
                "*"
              ],
              "output": "*"
            }
          ]
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataDestination": "data/filtered/thermostat"
        }
      }
    ]
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  }
}

Sample response

Status code:: 200

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "azure-iot-operations/data/thermostat"
          ]
        }
      },
      {
        "operationType": "BuiltInTransformation",
        "name": "transformation1",
        "builtInTransformationSettings": {
          "filter": [
            {
              "type": "Filter",
              "description": "filter-datapoint",
              "inputs": [
                "temperature.Value",
                "\"Tag 10\".Value"
              ],
              "expression": "$1 > 9000 && $2 >= 8000"
            }
          ],
          "map": [
            {
              "type": "PassThrough",
              "inputs": [
                "*"
              ],
              "output": "*"
            }
          ]
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataDestination": "data/filtered/thermostat"
        }
      }
    ],
    "provisioningState": "Succeeded"
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  },
  "id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
  "name": "jxhcpwgfkxqasbexkookvxk",
  "type": "zkuozvgjseokfchkscoswthzjdry",
  "systemData": {
    "createdBy": "ssvaslsmudloholronopqyxjcu",
    "createdByType": "User",
    "createdAt": "2024-08-09T18:13:29.389Z",
    "lastModifiedBy": "gnicpuszwd",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2024-08-09T18:13:29.389Z"
  }
}

Status code:: 201

Azure-AsyncOperation: https://contoso.com/operationstatus

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "azure-iot-operations/data/thermostat"
          ]
        }
      },
      {
        "operationType": "BuiltInTransformation",
        "name": "transformation1",
        "builtInTransformationSettings": {
          "filter": [
            {
              "type": "Filter",
              "description": "filter-datapoint",
              "inputs": [
                "temperature.Value",
                "\"Tag 10\".Value"
              ],
              "expression": "$1 > 9000 && $2 >= 8000"
            }
          ],
          "map": [
            {
              "type": "PassThrough",
              "inputs": [
                "*"
              ],
              "output": "*"
            }
          ]
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataDestination": "data/filtered/thermostat"
        }
      }
    ],
    "provisioningState": "Succeeded"
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  },
  "id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
  "name": "jxhcpwgfkxqasbexkookvxk",
  "type": "zkuozvgjseokfchkscoswthzjdry",
  "systemData": {
    "createdBy": "ssvaslsmudloholronopqyxjcu",
    "createdByType": "User",
    "createdAt": "2024-08-09T18:13:29.389Z",
    "lastModifiedBy": "gnicpuszwd",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2024-08-09T18:13:29.389Z"
  }
}

Dataflow_CreateOrUpdate_SimpleEventGrid

Sample request

HTTP

PUT https://management.azure.com/subscriptions/F8C729F9-DF9C-4743-848F-96EE433D8E53/resourceGroups/rgiotoperations/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/aio-to-event-grid?api-version=2024-08-15-preview

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "thermostats/+/telemetry/temperature/#"
          ]
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "event-grid-endpoint",
          "dataDestination": "factory/telemetry"
        }
      }
    ]
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  }
}

Sample response

Status code:: 200

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "thermostats/+/telemetry/temperature/#"
          ]
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "event-grid-endpoint",
          "dataDestination": "factory/telemetry"
        }
      }
    ],
    "provisioningState": "Succeeded"
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  },
  "id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
  "name": "jxhcpwgfkxqasbexkookvxk",
  "type": "zkuozvgjseokfchkscoswthzjdry",
  "systemData": {
    "createdBy": "ssvaslsmudloholronopqyxjcu",
    "createdByType": "User",
    "createdAt": "2024-08-09T18:13:29.389Z",
    "lastModifiedBy": "gnicpuszwd",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2024-08-09T18:13:29.389Z"
  }
}

Status code:: 201

Azure-AsyncOperation: https://contoso.com/operationstatus

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "thermostats/+/telemetry/temperature/#"
          ]
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "event-grid-endpoint",
          "dataDestination": "factory/telemetry"
        }
      }
    ],
    "provisioningState": "Succeeded"
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  },
  "id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
  "name": "jxhcpwgfkxqasbexkookvxk",
  "type": "zkuozvgjseokfchkscoswthzjdry",
  "systemData": {
    "createdBy": "ssvaslsmudloholronopqyxjcu",
    "createdByType": "User",
    "createdAt": "2024-08-09T18:13:29.389Z",
    "lastModifiedBy": "gnicpuszwd",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2024-08-09T18:13:29.389Z"
  }
}

Dataflow_CreateOrUpdate_SimpleFabric

Sample request

HTTP

PUT https://management.azure.com/subscriptions/F8C729F9-DF9C-4743-848F-96EE433D8E53/resourceGroups/rgiotoperations/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/aio-to-fabric?api-version=2024-08-15-preview

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "azure-iot-operations/data/thermostat"
          ]
        }
      },
      {
        "operationType": "BuiltInTransformation",
        "builtInTransformationSettings": {
          "serializationFormat": "Parquet",
          "schemaRef": "aio-sr://exampleNamespace/exmapleParquetSchema:1.0.0"
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "fabric-endpoint",
          "dataDestination": "telemetryTable"
        }
      }
    ]
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  }
}

Sample response

Status code:: 200

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "azure-iot-operations/data/thermostat"
          ]
        }
      },
      {
        "operationType": "BuiltInTransformation",
        "builtInTransformationSettings": {
          "serializationFormat": "Parquet",
          "schemaRef": "aio-sr://exampleNamespace/exmapleParquetSchema:1.0.0"
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "fabric-endpoint",
          "dataDestination": "telemetryTable"
        }
      }
    ],
    "provisioningState": "Succeeded"
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  },
  "id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
  "name": "jxhcpwgfkxqasbexkookvxk",
  "type": "zkuozvgjseokfchkscoswthzjdry",
  "systemData": {
    "createdBy": "ssvaslsmudloholronopqyxjcu",
    "createdByType": "User",
    "createdAt": "2024-08-09T18:13:29.389Z",
    "lastModifiedBy": "gnicpuszwd",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2024-08-09T18:13:29.389Z"
  }
}

Status code:: 201

Azure-AsyncOperation: https://contoso.com/operationstatus

{
  "properties": {
    "mode": "Enabled",
    "operations": [
      {
        "operationType": "Source",
        "name": "source1",
        "sourceSettings": {
          "endpointRef": "aio-builtin-broker-endpoint",
          "dataSources": [
            "azure-iot-operations/data/thermostat"
          ]
        }
      },
      {
        "operationType": "BuiltInTransformation",
        "builtInTransformationSettings": {
          "serializationFormat": "Parquet",
          "schemaRef": "aio-sr://exampleNamespace/exmapleParquetSchema:1.0.0"
        }
      },
      {
        "operationType": "Destination",
        "name": "destination1",
        "destinationSettings": {
          "endpointRef": "fabric-endpoint",
          "dataDestination": "telemetryTable"
        }
      }
    ],
    "provisioningState": "Succeeded"
  },
  "extendedLocation": {
    "name": "qmbrfwcpwwhggszhrdjv",
    "type": "CustomLocation"
  },
  "id": "/subscriptions/0000000-0000-0000-0000-000000000000/resourceGroups/resourceGroup123/providers/Microsoft.IoTOperations/instances/resource-name123/dataflowProfiles/resource-name123/dataflows/resource-name123",
  "name": "jxhcpwgfkxqasbexkookvxk",
  "type": "zkuozvgjseokfchkscoswthzjdry",
  "systemData": {
    "createdBy": "ssvaslsmudloholronopqyxjcu",
    "createdByType": "User",
    "createdAt": "2024-08-09T18:13:29.389Z",
    "lastModifiedBy": "gnicpuszwd",
    "lastModifiedByType": "User",
    "lastModifiedAt": "2024-08-09T18:13:29.389Z"
  }
}

Definitions

Name	Description
createdByType	The type of identity that created the resource.
DataflowBuiltInTransformationDataset	Dataflow BuiltIn Transformation dataset properties
DataflowBuiltInTransformationFilter	Dataflow BuiltIn Transformation filter properties
DataflowBuiltInTransformationMap	Dataflow BuiltIn Transformation map properties
DataflowBuiltInTransformationSettings	Dataflow BuiltIn Transformation properties
DataflowDestinationOperationSettings	Dataflow Destination Operation properties
DataflowMappingType	Dataflow type mapping properties
DataflowOperation	Dataflow Operation properties. NOTE - One only method is allowed to be used for one entry.
DataflowProperties	Dataflow Resource properties
DataflowResource	Instance dataflowProfile dataflow resource
DataflowSourceOperationSettings	Dataflow Source Operation properties
ErrorAdditionalInfo	The resource management error additional info.
ErrorDetail	The error detail.
ErrorResponse	Error response
ExtendedLocation	Extended location is an extension of Azure locations. They provide a way to use their Azure ARC enabled Kubernetes clusters as target locations for deploying Azure services instances.
ExtendedLocationType	The enum defining type of ExtendedLocation accepted.
FilterType	Filter Type properties
OperationalMode	Mode properties
OperationType	Dataflow Operation Type properties
ProvisioningState	The enum defining status of resource.
SourceSerializationFormat	Serialization Format properties
systemData	Metadata pertaining to creation and last modification of the resource.
TransformationSerializationFormat	Transformation Format properties

createdByType

The type of identity that created the resource.

Name	Type	Description
Application	string
Key	string
ManagedIdentity	string
User	string

DataflowBuiltInTransformationDataset

Dataflow BuiltIn Transformation dataset properties

Name	Type	Description
description	string	A user provided optional description of the dataset.
expression	string	Condition to enrich data from Broker State Store. Example: $1 < 0 \|\| $1 > $2 (Assuming inputs section $1 and $2 are provided)
inputs	string[]	List of fields for enriching from the Broker State Store.
key	string	The key of the dataset.
schemaRef	string	The reference to the schema that describes the dataset. Allowed: JSON Schema/draft-7.

DataflowBuiltInTransformationFilter

Dataflow BuiltIn Transformation filter properties

Name	Type	Default value	Description
description	string		A user provided optional description of the filter.
expression	string		Condition to filter data. Can reference input fields with {n} where n is the index of the input field starting from 1. Example: $1 < 0 \|\| $1 > $2 (Assuming inputs section $1 and $2 are provided)
inputs	string[]		List of fields for filtering in JSON path expression.
type	FilterType	Filter	The type of dataflow operation.

DataflowBuiltInTransformationMap

Dataflow BuiltIn Transformation map properties

Name	Type	Description
description	string	A user provided optional description of the mapping function.
expression	string	Modify the inputs field(s) to the final output field. Example: $1 * 2.2 (Assuming inputs section $1 is provided)
inputs	string[]	List of fields for mapping in JSON path expression.
output	string	Where and how the input fields to be organized in the output record.
type	DataflowMappingType	Type of transformation.

DataflowBuiltInTransformationSettings

Dataflow BuiltIn Transformation properties

Name	Type	Default value	Description
datasets	DataflowBuiltInTransformationDataset[]	[]	Enrich data from Broker State Store. Dataset references a key in Broker State Store.
filter	DataflowBuiltInTransformationFilter[]	[]	Filters input record or datapoints based on condition.
map	DataflowBuiltInTransformationMap[]	[]	Maps input to output message.
schemaRef	string		Reference to the schema that describes the output of the transformation.
serializationFormat	TransformationSerializationFormat	Json	Serialization format. Optional; defaults to JSON. Allowed value JSON Schema/draft-7, Parquet. Default: Json

DataflowDestinationOperationSettings

Dataflow Destination Operation properties

Name	Type	Description
dataDestination	string	Destination location, can be a topic or table name. Supports dynamic values with $topic, $systemProperties, $userProperties, $payload, $context, and $subscription.
endpointRef	string	Reference to the Endpoint CR. Can be of Broker, Kafka, Fabric, ADLS, ADX type.

DataflowMappingType

Dataflow type mapping properties

Name	Type	Description
BuiltInFunction	string	Built in function type
Compute	string	Compute type
NewProperties	string	New Properties type
PassThrough	string	Pass-through type
Rename	string	Rename type

DataflowOperation

Dataflow Operation properties. NOTE - One only method is allowed to be used for one entry.

Name	Type	Description
builtInTransformationSettings	DataflowBuiltInTransformationSettings	Built In Transformation configuration.
destinationSettings	DataflowDestinationOperationSettings	Destination configuration.
name	string	Optional user provided name of the transformation.
operationType	OperationType	Type of operation.
sourceSettings	DataflowSourceOperationSettings	Source configuration.

DataflowProperties

Dataflow Resource properties

Name	Type	Default value	Description
mode	OperationalMode	Enabled	Mode for Dataflow. Optional; defaults to Enabled.
operations	DataflowOperation[]		List of operations including source and destination references as well as transformation.
provisioningState	ProvisioningState		The status of the last operation.

DataflowResource

Instance dataflowProfile dataflow resource

Name	Type	Description
extendedLocation	ExtendedLocation	Edge location of the resource.
id	string	Fully qualified resource ID for the resource. E.g. "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/{resourceProviderNamespace}/{resourceType}/{resourceName}"
name	string	The name of the resource
properties	DataflowProperties	The resource-specific properties for this resource.
systemData	systemData	Azure Resource Manager metadata containing createdBy and modifiedBy information.
type	string	The type of the resource. E.g. "Microsoft.Compute/virtualMachines" or "Microsoft.Storage/storageAccounts"

DataflowSourceOperationSettings

Dataflow Source Operation properties

Name	Type	Default value	Description
assetRef	string		Reference to the resource in Azure Device Registry where the data in the endpoint originates from.
dataSources	string[]		List of source locations. Can be Broker or Kafka topics. Supports wildcards # and +.
endpointRef	string		Reference to the Dataflow Endpoint resource. Can only be of Broker and Kafka type.
schemaRef	string		Schema CR reference. Data will be deserialized according to the schema, and dropped if it doesn't match.
serializationFormat	SourceSerializationFormat	Json	Content is a JSON Schema. Allowed: JSON Schema/draft-7.

ErrorAdditionalInfo

The resource management error additional info.

Name	Type	Description
info	object	The additional info.
type	string	The additional info type.

ErrorDetail

The error detail.

Name	Type	Description
additionalInfo	ErrorAdditionalInfo[]	The error additional info.
code	string	The error code.
details	ErrorDetail[]	The error details.
message	string	The error message.
target	string	The error target.

ErrorResponse

Error response

Name	Type	Description
error	ErrorDetail	The error object.

ExtendedLocation

Extended location is an extension of Azure locations. They provide a way to use their Azure ARC enabled Kubernetes clusters as target locations for deploying Azure services instances.

Name	Type	Description
name	string	The name of the extended location.
type	ExtendedLocationType	Type of ExtendedLocation.

ExtendedLocationType

The enum defining type of ExtendedLocation accepted.

Name	Type	Description
CustomLocation	string	CustomLocation type

FilterType

Filter Type properties

Name	Type	Description
Filter	string	Filter type

OperationalMode

Mode properties

Name	Type	Description
Disabled	string	Disabled is equivalent to False.
Enabled	string	Enabled is equivalent to True

OperationType

Dataflow Operation Type properties

Name	Type	Description
BuiltInTransformation	string	Dataflow BuiltIn Transformation Operation
Destination	string	Dataflow Destination Operation
Source	string	Dataflow Source Operation

ProvisioningState

The enum defining status of resource.

Name	Type	Description
Accepted	string	Resource has been Accepted.
Canceled	string	Resource creation was canceled.
Deleting	string	Resource is Deleting.
Failed	string	Resource creation failed.
Provisioning	string	Resource is getting provisioned.
Succeeded	string	Resource has been created.
Updating	string	Resource is Updating.

SourceSerializationFormat

Serialization Format properties

Name	Type	Description
Json	string	JSON Format

systemData

Metadata pertaining to creation and last modification of the resource.

Name	Type	Description
createdAt	string	The timestamp of resource creation (UTC).
createdBy	string	The identity that created the resource.
createdByType	createdByType	The type of identity that created the resource.
lastModifiedAt	string	The timestamp of resource last modification (UTC)
lastModifiedBy	string	The identity that last modified the resource.
lastModifiedByType	createdByType	The type of identity that last modified the resource.

TransformationSerializationFormat

Transformation Format properties

Name	Type	Description
Delta	string	Delta Format
Json	string	JSON Format
Parquet	string	Parquet Format

Udostępnij za pośrednictwem

Dataflow - Create Or Update

URI Parameters

Request Body

Responses

Security

azure_auth

Scopes

Examples

Dataflow_CreateOrUpdate

Sample request

Sample response

Dataflow_CreateOrUpdate_ComplexContextualization

Sample request

Sample response

Dataflow_CreateOrUpdate_ComplexEventHub

Sample request

Sample response

Dataflow_CreateOrUpdate_FilterToTopic

Sample request

Sample response

Dataflow_CreateOrUpdate_SimpleEventGrid

Sample request

Sample response

Dataflow_CreateOrUpdate_SimpleFabric

Sample request

Sample response

Definitions

createdByType

DataflowBuiltInTransformationDataset

DataflowBuiltInTransformationFilter

DataflowBuiltInTransformationMap

DataflowBuiltInTransformationSettings

DataflowDestinationOperationSettings

DataflowMappingType

DataflowOperation

DataflowProperties

DataflowResource

DataflowSourceOperationSettings

ErrorAdditionalInfo

ErrorDetail

ErrorResponse

ExtendedLocation

ExtendedLocationType

FilterType

OperationalMode

OperationType

ProvisioningState

SourceSerializationFormat

systemData

TransformationSerializationFormat

Dodatkowe zasoby