Edit

Share via

Quickstart: Create a Stream Analytics job by using Visual Studio Code

In this quickstart, you create, run, and submit an Azure Stream Analytics (ASA) job by using the ASA Tools extension for Visual Studio Code on your local machine. You learn to build an ASA job that reads real-time streaming data from IoT Hub and filters events with a temperature greater than 27°. The output results are sent to a file in blob storage. The input data used in this quickstart is generated by a Raspberry Pi online simulator.

Note

Visual Studio Code tools don't support jobs in the China East, China North, Germany Central, and Germany Northeast regions.

Prerequisites

Install the Azure Stream Analytics Tools extension

  1. Open Visual Studio Code (VS Code).

  2. From Extensions on the left pane, search for stream analytics and select Install on the Azure Stream Analytics Tools extension.

    Screenshot showing the Extensions page of Visual Studio Code with an option to install Stream Analytics extension.

  3. After it's installed, select the Azure icon on the activity bar and sign in to Azure.

    Screenshot showing how to sign in to Azure.

  4. Once you're signed in, you can see the subscriptions under your Azure account.

Note

The ASA Tools extension automatically signs you in every time you open VS Code. If your account has two-factor authentication, use phone authentication rather than using a PIN. To sign out your Azure account, press Ctrl + Shift + P and enter Azure: Sign Out.

Prepare the input data

Before defining the Stream Analytics job, prepare the input data. The solution ingests the real-time sensor data to IoT Hub, which you later configure as the job input. To prepare the input data required by the job, complete the following steps:

  1. Sign in to the Azure portal.

  2. Select Create a resource. On the Create a resource page, select Internet of Things > IoT Hub.

    Screenshot showing the IoT Hub menu on the Create a resource page.

  3. On IoT Hub page, follow these steps:

    1. For Subscription, select your Azure subscription.
    2. For Resource group, select an existing resource group or create a new resource group.
    3. For IoT hub name, enter a name for your IoT hub.
    4. For Region, select the region that's closest to you.
    5. For Tier, select Free, if it's still available on your subscription. For more information, see IoT Hub pricing.
    6. For Daily message limit, keep the default value.
    7. Select Next: Networking at the bottom of the page.

    Screenshot showing the IoT Hub page for creation.

  4. Select Review + create. Review your IoT Hub information and select Create. Your IoT Hub might take a few minutes to create. You can monitor the progress in the Notifications pane.

  5. After the resource (IoT hub) is created, select Go to resource to go to the IoT Hub page.

  6. On the IoT Hub page, select Devices on the left menu, and then select + Add device.

    Screenshot showing the Add device button on the Devices page.

  7. Enter a Device ID and select Save.

    Screenshot showing the Create a device page.

  8. After you create the device, you see the device in the IoT devices list. Select Refresh button on the page if you don't see it.

    Screenshot showing the list of devices.

  9. Select your device from the list.

  10. On the device page, select the copy button next to Primary Connection String, and save it to a notepad to use later.

    Screenshot showing the copy button next to device connection string.

Run the IoT simulator

  1. Open the Raspberry Pi Azure IoT Online Simulator.

  2. Replace the placeholder in line 15 with the Azure IoT Hub device connection string you saved in a previous section.

  3. Select Run. The output shows the sensor data and messages that are sent to your IoT Hub.

    Screenshot showing the **Raspberry Pi Azure IoT Online Simulator** page with the sample query.

    Important

    Select Reset after a few minutes to reset the connection string.

Create blob storage

  1. From the upper-left corner of the Azure portal, select Create a resource > Storage > Storage account.

    Screenshot showing the Create storage account menu.

  2. In the Create storage account pane, enter a storage account name, location, and resource group. Choose the same location and resource group as the IoT hub that you created. Then select Review and Create to create the storage account.

    Screenshot showing the Create storage account page.

  3. On the Storage account page, select Containers on the left menu, and then select + Container on the command bar.

    Screenshot showing the Containers page.

  4. From the New container page, provide a name for your container, leave Public access level as Private (no anonymous access), and select OK.

    Screenshot showing the creation of a blob container page.

Create a Stream Analytics project

  1. In Visual Studio Code, select View > Command Palette, and enter ASA: Create New Project.

    Screenshot showing the selection of ASA: Create New Project in the command palette.

  2. Enter your project name, like myASAproj, and select a folder for your project.

    Screenshot showing entering an ASA project name.

  3. Visual Studio Code adds an ASA project to your workspace. It consists of three folders: Inputs, Outputs, and Functions. It also has the query script (*.asaql), a JobConfig.json file, and an asaproj.json configuration file.

    Screenshot showing Stream Analytics project files in Visual Studio Code.

    The asaproj.json file contains the inputs, outputs, and job configuration settings for submitting the Stream Analytics job to Azure.

    Note

    When you add inputs and outputs from the command palette, the corresponding paths are added to asaproj.json automatically. If you add or remove inputs or outputs on disk directly, you need to manually add or remove them from asaproj.json. You can put the inputs and outputs in one place and then reference them in different jobs by specifying the paths in each asaproj.json file.

Define the transformation query

  1. Open myASAproj.asaql file and add the following query:

    SELECT *
INTO Output
FROM Input
WHERE Temperature > 27

    Screenshot showing the transformation query.

Configure job input

  1. Right-click the Inputs folder in your Stream Analytics project. Then select ASA: Add Input from the context menu.

    Screenshot showing the ASA: Add input menu in Visual Studio Code.

    Or press Ctrl+Shift+P to open the command palette and enter ASA: Add Input.

  2. Choose IoT Hub for the input type.

    Screenshot showing the selection of your IoT hub in VS Code command palette.

  3. Choose Select from Azure subscriptions from the drop-down menu, and then press ENTER.

  4. Enter Input for the name, and then press ENTER.

  5. Under Inputs folder, you see an Input.json file is created.

  6. In the JSON file, confirm that Input is specified Name.

  7. In the JSON editor for Input.json, choose Select from your subscriptions, and then select your Azure subscription that has the IoT hub.

    Screenshot showing the JSON editor with Select from your subscriptions link.

  8. In the JSON editor, choose Select an IoT Hub, and then select the IoT hub you created.

    Screenshot showing the JSON editor with Select an IoT Hub link.

  9. By default, the SharedAccessPolicyName should be set to iothubowner. If not, choose Select a Shared Access Policy Name link, and then select iothubowner from the drop-down list.

  10. The SharedAccessPolicyKey value should be automatically set.

  11. Select Preview data to see if the input data is successfully configured for your job. It will fetch a sample of your IoT Hub and show in the preview window.

    Screenshot showing the preview of input data in your IoT hub.

Configure job output

  1. Right-click Outputs in the explorer, and select ASA: Add Output.
  2. Choose Data Lake Storage Gen2/Blob Storage for the sink type in the drop-down list.
  3. Choose Select from Azure subscriptions.
  4. Enter Output for the name of the alias and press ENTER. This output name is used for INTO statement in the query.
  5. In the JSON editor for Output.json, choose Select from your subscriptions, and then select your Azure subscription that has the Azure Storage account.
  6. If you need to change the storage account that's automatically filled, choose Select a Storage account, and then select your Azure Storage account. Storage account names are automatically detected if they're created in the same subscription.
  7. If you need to change the container name, choose Select a container, and select the blob container you created.

Screenshot showing the configuration of output for the Stream Analytics job.

Compile the script and submit it to Azure

Script compilation checks syntax and generates the Azure Resource Manager templates for automatic deployment.

  1. Right-click the script file in the explorer window, point to ASA: Compile Script, and then select ASA: ARM Template V2 (recommended).

    Screenshot of the compilation of script option from the Stream Analytics explorer in VS Code.

  2. After compilation, you see a Deploy folder under your project with two Azure Resource Manager templates. These two files are used for automatic deployment.

    Screenshot of the generated deployment templates in the project folder.

  3. Select Submit to Azure in the query editor.

    Screenshot of the Submit job button to submit the Stream Analytics job to Azure.

  4. In the Submit window, follow these steps:

    1. Select your Azure subscription.

    2. Select an Azure resource group.

    3. Select the region where you want to create the Stream Analytics job.

    4. Then, select Submit.

      Screenshot of Submit options.

  5. Select Publish to Azure and complete. Wait for it to open a new tab Cloud Job View showing your job's status.

    Screenshot of the publish to Azure button in VS Code.

Start the Stream Analytics job and check output

  1. On the Cloud Job View tab, select Start to run your job in the cloud.

    Screenshot of the Start job button in the Cloud view page.

  2. On the Start streaming job window, select OK. This process might take a few minutes to complete.

    Screenshot of the Start streaming job window.

  3. If your job starts successfully, the job status changes to Running. You see a logical diagram showing how your ASA job is running.

    Screenshot of the job running status in VS Code.

  4. To view the output results, open the blob storage in the Visual Studio Code extension or in the Azure portal.

    Screenshot of the output file in the Blob container.

    Download and open the file to see output.

    {"messageId":11,"deviceId":"Raspberry Pi Web Client","temperature":28.165519323167562,"humidity":76.875393581654379,"EventProcessedUtcTime":"2022-09-01T22:53:58.1015921Z","PartitionId":3,"EventEnqueuedUtcTime":"2022-09-01T22:52:57.6250000Z","IoTHub":{"MessageId":null,"CorrelationId":null,"ConnectionDeviceId":"MyASAIoTDevice","ConnectionDeviceGenerationId":"637976642928634103","EnqueuedTime":"2022-09-01T22:52:57.6290000Z"}}
{"messageId":14,"deviceId":"Raspberry Pi Web Client","temperature":29.014941877871451,"humidity":64.93477299527828,"EventProcessedUtcTime":"2022-09-01T22:53:58.2421545Z","PartitionId":3,"EventEnqueuedUtcTime":"2022-09-01T22:53:03.6100000Z","IoTHub":{"MessageId":null,"CorrelationId":null,"ConnectionDeviceId":"MyASAIoTDevice","ConnectionDeviceGenerationId":"637976642928634103","EnqueuedTime":"2022-09-01T22:53:03.6140000Z"}}
{"messageId":17,"deviceId":"Raspberry Pi Web Client","temperature":28.032846241745975,"humidity":66.146114343897338,"EventProcessedUtcTime":"2022-09-01T22:53:58.2421545Z","PartitionId":3,"EventEnqueuedUtcTime":"2022-09-01T22:53:19.5960000Z","IoTHub":{"MessageId":null,"CorrelationId":null,"ConnectionDeviceId":"MyASAIoTDevice","ConnectionDeviceGenerationId":"637976642928634103","EnqueuedTime":"2022-09-01T22:53:19.5830000Z"}}
{"messageId":18,"deviceId":"Raspberry Pi Web Client","temperature":30.176185593576143,"humidity":72.697359909427419,"EventProcessedUtcTime":"2022-09-01T22:53:58.2421545Z","PartitionId":3,"EventEnqueuedUtcTime":"2022-09-01T22:53:21.6120000Z","IoTHub":{"MessageId":null,"CorrelationId":null,"ConnectionDeviceId":"MyASAIoTDevice","ConnectionDeviceGenerationId":"637976642928634103","EnqueuedTime":"2022-09-01T22:53:21.6140000Z"}}
{"messageId":20,"deviceId":"Raspberry Pi Web Client","temperature":27.851894248213021,"humidity":71.610229530268214,"EventProcessedUtcTime":"2022-09-01T22:53:58.2421545Z","PartitionId":3,"EventEnqueuedUtcTime":"2022-09-01T22:53:25.6270000Z","IoTHub":{"MessageId":null,"CorrelationId":null,"ConnectionDeviceId":"MyASAIoTDevice","ConnectionDeviceGenerationId":"637976642928634103","EnqueuedTime":"2022-09-01T22:53:25.6140000Z"}}
{"messageId":21,"deviceId":"Raspberry Pi Web Client","temperature":27.718624694772238,"humidity":66.540445035685153,"EventProcessedUtcTime":"2022-09-01T22:53:58.2421545Z","PartitionId":3,"EventEnqueuedUtcTime":"2022-09-01T22:53:48.0820000Z","IoTHub":{"MessageId":null,"CorrelationId":null,"ConnectionDeviceId":"MyASAIoTDevice","ConnectionDeviceGenerationId":"637976642928634103","EnqueuedTime":"2022-09-01T22:53:48.0830000Z"}}
{"messageId":22,"deviceId":"Raspberry Pi Web Client","temperature":27.7849054424326,"humidity":74.300662748167085,"EventProcessedUtcTime":"2022-09-01T22:54:09.3393532Z","PartitionId":3,"EventEnqueuedUtcTime":"2022-09-01T22:54:09.2390000Z","IoTHub":{"MessageId":null,"CorrelationId":null,"ConnectionDeviceId":"MyASAIoTDevice","ConnectionDeviceGenerationId":"637976642928634103","EnqueuedTime":"2022-09-01T22:54:09.2400000Z"}}
{"messageId":28,"deviceId":"Raspberry Pi Web Client","temperature":30.839892925680324,"humidity":76.237611741451786,"EventProcessedUtcTime":"2022-09-01T22:54:47.8053253Z","PartitionId":3,"EventEnqueuedUtcTime":"2022-09-01T22:54:47.6180000Z","IoTHub":{"MessageId":null,"CorrelationId":null,"ConnectionDeviceId":"MyASAIoTDevice","ConnectionDeviceGenerationId":"637976642928634103","EnqueuedTime":"2022-09-01T22:54:47.6150000Z"}}
{"messageId":29,"deviceId":"Raspberry Pi Web Client","temperature":30.561040300759053,"humidity":78.3845172058103,"EventProcessedUtcTime":"2022-09-01T22:54:49.8070489Z","PartitionId":3,"EventEnqueuedUtcTime":"2022-09-01T22:54:49.6030000Z","IoTHub":{"MessageId":null,"CorrelationId":null,"ConnectionDeviceId":"MyASAIoTDevice","ConnectionDeviceGenerationId":"637976642928634103","EnqueuedTime":"2022-09-01T22:54:49.5990000Z"}}
{"messageId":31,"deviceId":"Raspberry Pi Web Client","temperature":28.163585438418679,"humidity":60.0511571297096,"EventProcessedUtcTime":"2022-09-01T22:55:25.1528729Z","PartitionId":3,"EventEnqueuedUtcTime":"2022-09-01T22:55:24.9050000Z","IoTHub":{"MessageId":null,"CorrelationId":null,"ConnectionDeviceId":"MyASAIoTDevice","ConnectionDeviceGenerationId":"637976642928634103","EnqueuedTime":"2022-09-01T22:55:24.9120000Z"}}
{"messageId":32,"deviceId":"Raspberry Pi Web Client","temperature":31.00503387156985,"humidity":78.68821066044552,"EventProcessedUtcTime":"2022-09-01T22:55:43.2652127Z","PartitionId":3,"EventEnqueuedUtcTime":"2022-09-01T22:55:43.0480000Z","IoTHub":{"MessageId":null,"CorrelationId":null,"ConnectionDeviceId":"MyASAIoTDevice","ConnectionDeviceGenerationId":"637976642928634103","EnqueuedTime":"2022-09-01T22:55:43.0520000Z"}}

Clean up resources

When you no longer need the resources, delete the resource group, the Stream Analytics job, and all related resources. Deleting the job stops billing for the streaming units the job consumes. If you plan to use the job in the future, you can stop it and restart it later when you need. If you aren't going to continue to use this job, delete all resources created by this quickstart by using the following steps:

  1. From the left menu in the Azure portal, select Resource groups and then select the name of the resource that you created.

  2. On your resource group page, select Delete. Enter the name of the resource to delete in the text box, and then select Delete.

Next steps

To learn more about ASA Tools extension for Visual Studio Code, see the following articles:

  • Last updated on