Build your object detection custom model

Article
01/10/2024

Sign in to Power Apps or Power Automate.
On the left pane, Select ... More > AI hub.
Under Discover an AI capability, select AI models.

(Optional) To keep AI models permanently on the menu for easy access, select the pin icon.
Select Images > Object detection - Detect custom objects in images.
Select Create custom model.

Select the model domain

The first thing you'll do when you create an AI Builder object detection model is to define its domain. The domain optimizes the model for specific use cases. There are three domains:

Common objects: The default value. Use this if your use case doesn't fit the specific applications below.
Objects on retail shelves: Detects products densely packed on shelves.
Brand logos: Optimized for logo detection.

Select a model for your domain and select Next.

Note

Domain specific object detection uses more AI Builder credits than common object detection, and therefore costs more to use. To estimate the impact of using domain specific versus common object on your organization, use the AI Builder calculator.

Provide object names

Next, provide the names of the items you want to detect. You can provide up to 500 object names per model.

There are two ways to provide object names:

Enter object names directly in AI Builder.
Select names from your Microsoft Dataverse table.

Important

If you change input mode, you lose any existing object names. In other words, if you type object names in AI Builder and then change to selecting from a database, all object names typed and their associated bounding boxes are deleted from your model. This is not recoverable. The same applies if you switch from selecting object names from a database to typing object names.

To choose objects from a Dataverse table, choose Select from database above Choose objects for your model to detect, and then choose Select object names. If you change your mind before you select your table, you can select Add objects manually to switch back.

Enter names in AI Builder

To provide object names directly in AI Builder, just enter the name in the space where the object is detected in the image. Then press Enter or select Add new object to continue.

To edit an object name, select the object name and then make your change.
To remove an object name, select the trashcan icon.

Select names from a database

If your data isn't in Dataverse, go to Prerequisites for information about how to import data into Dataverse.

Choose Select from database to see tables in your environment.
On the panel to the right, find and select the table that contains your object names.
Select the column that contains the names of your objects, and then choose Select column.
From the list of objects in your table, select the ones that represent the objects you want to detect.
At the bottom of the screen, select Next.

Upload images

Now let's move on to the image upload step. The pictures you collected ahead of time will now come in handy because you need to upload them to AI Builder.

Prepare your images in the storage location where you want to add them from. Currently you can add images from local storage, SharePoint, or Azure Blob Storage.
Make sure your images follow the qualitative and quantitative guidance.
In AI Builder, select Add images.
Select the data source where your images are stored, and then select the images that contain your objects.
Confirm the images that appear in AI Builder before you upload them. Deselect any image you want to exclude.
Select Upload <number> images.
When the upload is complete, select Close, and then select Next.

Tag images

This section explains the tagging process that's a key part of object detection. You draw rectangles around the objects of interest, and then assign a name to the rectangle that you want the model to associate with this object.

On the Tag the objects in your images screen, select the first image in your gallery.
Draw a rectangle around the object. To do this, press and hold your mouse at the upper-left corner of the object, and then drag down to the lower-right corner of the object. The rectangle should fully encompass the object you want your model to recognize.
After you draw a rectangle, you can associate a name to the object from the list of names you already selected.
Your tag is created when you see it surrounding an object.
Navigate from image to image, and tag at least 15 images per object name to build a model.
After you're done tagging your images, select Done tagging. Your data is saved as you create rectangles.
In the grid view, you can view a summary of all the tags you created and which images you created. This lets you know how much more work is needed to move forward.
Until you reach the minimum for content quantity, you can't move forward. After you have at least 15 images per object name, you'll be able to select Next at the bottom of the screen.