Configure the connector for HTTP/REST

In Azure IoT Operations, the connector for HTTP/REST enables access to data from REST endpoints exposed by HTTP services.

An asset in Azure IoT Operations is a logical entity that you create to represent a physical asset or device. An Azure IoT Operations asset can have custom properties, data points, streams, and events that describe its behavior and characteristics. An asset is associated with one or more devices. Azure IoT Operations stores asset definitions in the Azure Device Registry.

A device in Azure IoT Operations is a logical entity that defines the connections to physical assets or devices. Without a device, data can't flow from a physical device or asset to the MQTT broker. When you configure a device and asset, a connection is established to the physical asset or device and data point values, events, and streams arrive in Azure IoT Operations instance. A device has one or more inbound endpoints. Azure IoT Operations stores device definitions in the Azure Device Registry.

The connector for HTTP/REST supports the following features:

• Automatic retries when sampling failures occur. Reports a failed status for errors that can't be retried.
• Integration with OpenTelemetry.
• Use of device endpoints and assets.
• Optionally transform incoming data using WASM modules.
• Device endpoint and asset definition validation for REST compatibility.
• Multiple authentication methods:
  • Username/password basic HTTP authentication
  • x509 client certificates
  • Anonymous access for testing purposes
• To establish a TLS connection to the HTTP endpoint, you can configure a certificate trust list for the connector.

For each configured dataset, the connector for HTTP/REST:

• Performs a GET request to the address specified in the device endpoint and appends the dataset's data source from the asset.
• Generates a message schema for each dataset based on the data it receives, and registers it with Schema Registry and Azure Device Registry.
• Forwards the data to the specified destination.

This article explains how to use the connector for HTTP/REST to perform tasks such as:

• Define the devices that connect HTTP sources to your Azure IoT Operations instance.
• Add assets, and define the data points to enable the data flow from the HTTP source to the MQTT broker or broker state store.

Prerequisites

To configure devices and assets, you need a running instance of Azure IoT Operations.

To sign in to the operations experience web UI, you need a Microsoft Entra ID account with at least contributor permissions for the resource group that contains your Kubernetes - Azure Arc instance. You can't sign in with a Microsoft account (MSA). For more information, see Troubleshoot access to the operations experience web UI.

Your IT administrator must configure the connector for HTTP/REST template for your Azure IoT Operations instance in the Azure portal.

You need any credentials required to access the HTTP source. If the HTTP source requires authentication, you need to create a Kubernetes secret that contains the username and password for the HTTP source.

Deploy the connector for HTTP/REST

When you deploy Azure IoT Operations, the deployment includes various connectors. Before you can use the connectors (such as ONVIF, media, and HTTP/REST) in the operations experience web UI, an administrator must add connector template instances to your Azure IoT Operations instance.

All the connectors can publish captured data to the MQTT broker.

To add a connector template instance to your Azure IoT Operations instance:

1. In the Azure portal, go to your Azure IoT Operations instance, select Connector templates, and then select Add connector template instances:

   

2. On the first page of the Add an Akri connector template wizard, select the type and version of connector template you want to add, such as ONVIF, Media, HTTP/REST, SSE, or MQTT. Then select Metadata.

   

3. On the Metadata page, accept the defaults, and then select Device inbound endpoint type.

4. On the Device inbound endpoint type page, accept the defaults, and then select Diagnostics configurations.

5. On the Diagnostics configurations page, accept the defaults, and then select Runtime configuration.

6. On the Runtime configuration page, accept the defaults, and then select Review.

7. On the Review page, review the details of the connector template instance, and then select Create to create the connector template instance.

An OT user can now use the operations experience web UI to create a device with a connector endpoint.

---

Create a device

To configure the connector for HTTP/REST, first create a device that defines the connection to the HTTP source. The device includes the URL of the HTTP source and any credentials you need to access the HTTP source:

Operations experience

1. In the operations experience web UI, select Devices in the left navigation pane. Then select Create new.

2. Enter a name for your device, such as http-connector. To add the endpoint for the connector for HTTP/REST, select New on the Microsoft.Http tile.

3. Add the details of the endpoint for the connector for HTTP/REST including any authentication credentials:

   

   Select Apply to save the endpoint.

4. On the Device details page, select Next to continue.

5. On the Add custom property page, add any other properties you want to associate with the device. For example, you might add a property to indicate the manufacturer of the camera. Then select Next to continue.

6. On the Summary page, review the details of the device and select Create to create the asset.

7. After the device is created, you can view it in the Devices list:

   

Azure CLI

Run the following commands:

az iot ops ns device create -n rest-http-connector-cli -g {your resource group name} --instance {your instance name} 

az iot ops ns device endpoint inbound add rest --device rest-http-connector-cli -g {your resource group name} -i {your instance name}  --name rest-http-connector-0 --endpoint-address "https://rest-http-connector-0"

To learn more, see az iot ops ns device.

Configure a device to use a username and password

The previous example uses the Anonymous authentication mode. This mode doesn't require a username or password.

To use the Username password authentication mode, complete the following steps:

Operations experience

Follow the steps in Manage secrets for your Azure IoT Operations deployment to add secrets for username and password in Azure Key Vault, project them into Kubernetes cluster, and reference them from your device configuration.

Azure CLI

1. Follow the steps in Manage secrets for your Azure IoT Operations deployment to add secrets for username and password in Azure Key Vault, and project them into Kubernetes cluster.

2. Use the az iot ops ns device endpoint inbound add command with the --user-ref and --pass-ref parameters to add the references to the Azure Key Vault secrets you created.

Configure a device to use an X.509 certificate

To use the X509 certificate authentication mode, follow the steps in Manage certificates for external communications to add secrets for certificates in Azure Key Vault, project them into Kubernetes cluster, and reference them from your device configuration.

Configure a certificate trust list for a device to use

To manage the trusted certificates list for the connector for HTTP/REST, see Manage certificates for external communications.

Create an asset

To define an asset that publishes data points from the HTTP endpoint, follow these steps:

Operations experience

1. In the operations experience web UI, select Assets in the left navigation pane. Then select Create asset.

2. Select the inbound endpoint for the connector for HTTP/REST that you created in the previous section.

3. Enter a name for your asset, such as my-http-source.

4. Add any custom properties you want to associate with the asset. For example, you might add a property to indicate the manufacturer of the camera. Select Next to continue.

A dataset defines where the connector sends the data it collects from a collection of data points. An HTTP/REST asset can have multiple datasets. To create a dataset:

1. Select Create dataset.

2. Enter the details for the dataset such as its name, data source, sampling interval, and destination. For HTTP/REST assets, the data source is the path on the REST endpoint. For HTTP/REST assets, the destination is either an MQTT topic or a broker state store key. For example:

   

   To transform the incoming data, add the URL of a WebAssembly (WASM) module in the Transform field. To learn more, see Transform incoming data.

3. Select Create and next to create the dataset.

   Tip

   Use the Manage default settings option to configure default dataset settings such as the sampling interval.

4. On the Review page, review the details of the asset and select Create to create the asset. After a few minutes, the asset is listed on the Assets page:

   

Azure CLI

Run the following commands:

az iot ops ns asset rest create --name myrestasset --instance {your instance name} -g {your resource group name} --device rest-http-connector-cli --endpoint rest-http-connector-0

az iot ops ns asset rest dataset add --asset myrestasset --instance {your instance name} -g {your resource group name} --name weatherdata --data-source "/api/weather" --dest topic="azure-iot-operations/data/erp" retain=Never qos=Qos1 ttl=3600 --sampling-int 30000

For more information, see az iot ops ns asset rest.

Transform incoming data

To transform the incoming data by using a WASM module, complete the following steps:

1. Develop a WASM module to perform the custom transformation. For more information, see Develop WebAssembly (WASM) modules and graph definitions.

2. Configure your transformation graph. For more information, see Configure WebAssembly (WASM) graph definitions.

3. Deploy both the module and graph. For more information, see Use WebAssembly (WASM).

   Note

   You need to deploy at least one data flow graph to enable WASM graph processing, but this feature doesn't otherwise use the graph.

4. Configure your dataset with the URL of the deployed WASM graph in the Transform field:

   

A data transformation in the HTTP/REST connector only requires a single map operator, but WASM graphs are fully supported with the following restrictions:

• The graph must have a single source node and a single sink node.
• The graph must consume and emit the DataModel::Message datatype.
• The graph must be stateless. Currently, this restriction means that accumulate operators aren't supported.