Create a people counting solution with Azure Percept Vision
Retirement of Azure Percept DK:
Update November 9, 2022: A firmware update that enables the Vision SoM and Audio SOM to retain their functionality with the DK beyond the retirement date, will be made available before the retirement date.
The Azure Percept public preview will be evolving to support new edge device platforms and developer experiences. As part of this evolution the Azure Percept DK and Audio Accessory and associated supporting Azure services for the Percept DK will be retired March 30th, 2023.
Effective March 30th, 2023, the Azure Percept DK and Audio Accessory will no longer be supported by any Azure services including Azure Percept Studio, OS updates, containers updates, view web stream, and Custom Vision integration. Microsoft will no longer provide customer success support and any associated supporting services. For more information, please visit the Retirement Notice Blog Post.
This guide will focus on detecting and counting people using the Azure Percept DK hardware, Azure IoT Hub, Azure Stream Analytics, and Power BI dashboard.
The tutorial is intended to show detailed steps on how users can create, configure, and implement the basic components of this solution. Users can easily expand the tutorial and create additional ways to visualize people counting data.
Top customer scenarios:
- People counting intelligence: aggregation of people counting over a given day, week, or duration.
- Occupancy: determine when a space is free and available for use. Quantify how long the space is idle and unused.
- Understanding peak occupancy levels and when they occur.
- Detecting people counting after hours: count of people in space during non-business hours.
In this tutorial, you learn how to:
- Set up your Azure Percept DK and Vision AI model
- Create a Container Registry resource
- Build and push your edge solution to Container Registry
- Deploy edge solution to device
- Add a consumer group to your IoT Hub
- Create a Stream Analytics Job
- Create and publish a Power BI report to visualize data
Input : Video stream from Azure Percept DK
Output: Count of people in Power BI dashboard
Azure Subscription: (Free trial account)
Power BI subscription: (Try Power BI for free)
Power BI workspace: (Create the new workspaces in Power BI)
Azure Percept DK setup experience: you connected your devkit to a Wi-Fi network, created an IoT Hub, and connected your devkit to the IoT Hub
Download and install VS Code
Download and install Git
Install the IoT Hub Extension in VS Code
Install the Azure IoT Tools Extension in VS Code
Download and install Docker Desktop (Will require a PC reboot)
(Only for Windows Users) Install WSL2 by running the following commands in Windows PowerShell or Terminal (on macOS) (Will require a PC restart)
wsl --set-default-version 2
Step 0: Set up your Azure Percept DK and Vision AI model
Setting up the Azure Percept DK is the first step in the tutorial. Below are the steps to follow and links to further guidance.
Follow Quickstart: unbox and assemble your Azure Percept DK components and the next steps.
Connect the camera module to the Azure Percept DK via the USB-C cable.
Open Command Prompt (for Windows) or Terminal (on macOS) and execute the command-
git clone https://github.com/microsoft/Azure-Percept-Reference-Solutions.git
Within the cloned repository go to
Step 1: Create a Container Registry resource
Azure Container Registry is a managed, private Docker registry service based on the open-souce Docker Registry. Container Registries are used to manage and store your private Docker containers images and related artifacts.
Login to Azure portal https://portal.azure.com/
To create a Container Registry, go to Create container registry - Microsoft Azure
a. Select your Azure Subscription in the
b. Select your preferred resource group from the
Resource groupdrop-down menu. It is recommended to use the
Resource groupwhich contains the IOT Hub connected to the Azure Percept DK.
c. Provide a unique
Location, select the region to deploy resource (We suggest select
Availability Zones- disabled
g. Keep all other tab as default and click
Review + createat the bottom of the screen. Once the validation passes, click
Create. This will create your Container Registry.
After successful resource deployment go to your container registry resource. On the left scroll panel select
Make a note of the
Go to the git repository and
people-counting-with-azure-percept-visiondirectory and rename
.env. Open the file and fill in the following details-
a. CONTAINER_REGISTRY_USERNAME= your container registry Username
b. CONTAINER_REGISTRY_PASSWORD= your container registry Password
c. CONTAINER_REGISTRY_LOGINSERVER= your container registry Login Server
Step 2: Build and push your edge solution to Container Registry
This section guides users on modifying the cloned people counting repo with their individual deployment information, building the model image, and pushing model image to container registry.
Open VS Code, at the bottom of the screen ensure you have
Default Platform for IoT Edge Solutionselected (if not, then please click and select arm64v8 from the list)
people-counting-with-azure-percept-visiondirectory go to
modules/CountModule/directory and open
module.json. Fill in your
Container registry address(same as the
Login serversaved earlier) and followed by a
repository name(Note- please make sure your repository name is all lowercase)
"repository": "<Your container registry login server/repository name>"
will change as follows, for example-
Now you will build the module image and push it to your container registry. Open Visual Studio Code integrated terminal by selecting
View > Terminal
Sign into Docker with the Azure Container registry (ACR) credentials that you saved after creating the registry using below command in terminal. Note that this command would give a warning that using --password or -p via CLI is insecure. Therefore, if you want a more secure login for your future solution development, use
--password-stdininstead by following this instruction.
docker login -u <ACR username> -p <ACR password> <ACR login server>
Visual Studio Code now has access to your container registry. In the next steps you will turn the solution code into a container image. In Visual Studio Code explorer, right click the
deployment.template.jsonfile and select
Build and Push IoT Edge Solution
The build and push command starts three operations. First, it creates a new folder in the solution called
configthat holds the full deployment manifest, built out of information in the deployment template and other solution files. Second, it runs
docker buildto build the container image based on the appropriate docker file for your target architecture. Then, it runs
docker pushto push the image repository to your container registry. This process may take several minutes the first time but is faster the next time that you run the commands.
deployment.arm64v8.jsonfile in the newly created config folder. The filename reflects the target architecture, so it will be different if you choose a different architecture.
Notice that the two parameters that had placeholders now are filled in with their proper values. The
registryCredentialssection has your registry username and password pulled from the .env file. The
CountModulehas the full image repository with the
architecturetag from the
To further verify what the build and push command did, go to the Azure portal, and navigate to your container registry. In your container registry, select
Step 3: Deploy edge solution to device
Step 3 will guide users on creating and deploying a manifest to the Azure Percept Dev Kit. This deployment will create a new edge module ‘CountModule’ and will overwrite any previous deployments of ‘CountModule’.
In the Visual Studio Code explorer, under the
Azure IoT Hubsection, expand
Devicesto see your list of IoT devices
Right-click the IoT Edge device that you want to deploy to, then select
Create Deployment for Single Device
In the file explorer, navigate into the
configfolder then select the
deployment.arm64v8.jsonfile and click
Select Edge Deployment Manifest.
Do not use the deployment.template.json file, which does not have the container registry credentials or module image values in it.
Under your device, expand
Modulesto see a list of deployed and running modules. Click the refresh button. You should see the
CountModulerunning on your device.
Go to Azure Percept Studio and on the left panel, select Devices, then select your Azure Percept device
Ensure that your device is
Connected. Click on
View your device stream
The previous step will deploy modules to your device. In the
View Stream. This will open a new tab in your browser, please verify that you see the video stream. If you point the camera module to a person then you will see the person detection with bounding box
After verifying the video stream and bounding boxes, please close the web stream browser tab.
To ensure the Count Module is setup correctly, in the Azure portal go to your IoT Hub. On the left panel under
From the IoT device list click on your Azure Percept DK device
Scroll down to check if all deployed modules are in
From the drop-down list select
Ensure you see
People_Countlogs as follows-
Step 4: Add a consumer group to your IoT Hub
Consumer Groups provide independent views into the event stream that enable apps and Azure services to independently consume data. This consumer group will be used by the Stream Analytics Job we will create in Step 5.
In the Azure portal, go to your IoT hub which is connected to your Azure Percept DK.
On the left pane, select
Hub settings > Built-in endpoints. Enter a name for your new consumer group in the text box under
Click anywhere outside the text box to save the consumer group
Step 5: Create a Stream Analytics job
Step 5 guides users through creating, configuring, and running a Stream Analytics job. Stream Analytics is a hot path to stream data from out Azure IoT Hub to a Power BI workspace in real time. We will create a query so only People Counting telemetry will be streamed. Once People Counting data is in our Power BI workspace it will be easy to render with a Power BI report.
Go to New Stream Analytics job - Microsoft Azure
Enter the following information for the job -
Job name- The name of the job. The name must be globally unique.
Resource group- Use the same resource group that your IoT hub uses.
Location- Use the same location as your resource group.
Add an input to the Stream Analytics job
Open the previously created Stream Analytics job. Under
Job topology, select
Add stream input, then select
IoT Hubfrom the drop-down list.
On the new input pane, enter the following information -
Input alias- Enter a unique alias for the input
Select IoT Hub from your subscription- Select this radio button
Subscription- Select the Azure subscription you are using for this lab
IoT Hub- Select the IoT Hub you are using for this lab
Consumer group- Select the consumer group you created previously
Shared access policy name- Select the name of the shared access policy you want the Stream Analytics job to use for your IoT hub. For this lab, you can select service
Shared access policy key- This field is auto filled based on your selection for the shared access policy name
Endpoint- Select Messaging
Leave all other fields as default-
Add an output to the Stream Analytics job
Create a Group Workspace, take the following steps to create one -
a. In a new web browser tab open Power BI
b. On the left panel click on
Workspaces > Create a workspace
c. Give your workspace a name and description (optional) and click
d. Go back to the Azure portal and go to the Stream Analytics job
Job topology, select
Add, and then select
Power BIfrom the drop-down list
Enter the following information-
Output alias- A unique alias for the output
Select Group workspace from your subscriptions- Select this radio button
Group workspace- Select your target group workspace
Dataset name- Enter a dataset name
Table name- Enter a table name
Authentication mode- User token
Power BI - New outputpane, select
Authorizeand follow the prompts to sign into your Power BI account
Configure the query of the Stream Analytics job
Job topology, select
[YourInputAlias]with the input alias of the job
[YourOutputAlias]with the output alias of the job
Add the following
WHEREclause as the last line of the query. This line ensures that only messages with a
People_Countproperty will be forwarded to Power BI.
WHERE People_Count IS NOT NULL
The query will look as follows -
People_Countproperty is sent from the
countmoduleto the IoT hub and is forwarded to the Stream Analytics job.
Run the Stream Analytics job
In the Stream Analytics job, select
Overview, then select
Start > Now > Start
Once the job successfully starts, the job status changes from
Step 6: Create and publish a Power BI report to visualize data
This step will guide users on how to create a Power BI report from the People Counting telemetry data. The tutorial walks through initial steps to visualize people counting data. Users who are interested to learn more ways to transform, aggregate, and visualize their data could explore the Power BI product page for ideas and templates.
Login to Power BI and select your Workspace (this is the same Group Workspace you used while creating the Stream Analytics job output)
Verify that you see your dataset
On the left scroll panel select
+ Createand then click
Pick a published dataset
Select your dataset and click
On the right, expand the
Fieldsdropdown and select
Line and clustered column chart
This will generate a graph as follows-
Refreshperiodically to update the graph
Step 7: Clean up resources
If you're not going to continue to use this application, delete Azure resources with the following steps:
Login to the Azure portal, go to
Resource Groupyou have been using for this tutorial. Select the
Stream Analytics Jobresource created and stop the job from running then delete.
Login to Power BI and select your Workspace (this is the same Group Workspace you used while creating the Stream Analytics job output), and delete workspace.
Check out the other tutorial under Advanced prototyping with Azure Percept section for your Azure Percept DK.