Custom Vision workflow question regarding Tagging and Training during object detection

Question

Custom Vision workflow question regarding Tagging and Training during object detection

Ryan 21

I'm super new to using Custom Vision, specifically the object recognition. I have a question regarding the best workflow and a couple weirdly specific questions. Any input would be appreciated!

When tagging images with the intent to use them for your next Iteration, is there any difference between the following flows:
--Upload via "Add Images", they will be in untagged. Tag them from Untagged
or
--Quick Test, and then view that image's results in the Predictions tab. Then add/edit/correct all tags here, and when finished it will automatically move to the Tagged bin

Obviously Quick Test shows you some preview results. But besides seeing results, does that second flow (using Quick Test) add any extra umph to the training/tags/process in any way at all?

Does it matter how the images end up in the Trained folder? Is one route more helpful in training than the other route.

--

Would a good practice be to add a range of images to Quick Test / Predictions tab....and then just leave them there in the Prediction tab (untouched) to observe the results of those images on each future Training Iteration? .....Does each iteration require new Quick Tests to be run or can the existing ones on the Prediction tab just be applied to the a new iteration?

Thanks in advance for any input at all!

Ryan 21 Reputation points

2023-01-05T23:23:00+00:00

Hi @Ramr-msft , thank you for the time and help so far. I wanted to ask a couple more questions, and was hoping it was OK to just keep going in this thread. If it helps, I can re-post this as a brand new question if you would rather me do it that way.

1) I'm wondering what impact (if any) the size of the objects will have during the training and the AI recognition process. For example, we want to recognize people on cellphones in a fairly distant crowd. All of the recognition will be from the same somewhat far distance if that makes sense. (The subjects we are trying to recognize will always be roughly the same size in the images due to a fixed distance from the camera.)

So when we use images to train, or quick test, or ultimately run through the final iteration....the "person holding a cellphone" will always be roughly an inch tall on my monitor. When we train to recognize this scenario (specifically, little 1-inch people with phones here and there), does it help to train with a closeup picture of a person on a phone, that's maybe 6-8-10 inches tall on my screen? Should we try to train it to recognize the objects regardless of size? Or would it be better to try and train it with appropriately sized objects? Any thoughts you have are welcome and appreciated, thank you.
Ryan 21 Reputation points

2023-01-05T23:30:50.07+00:00

2) To increase recognition accuracy when recognizing "a person on a cellphone", which one of the following scenarios would be more accurate:

A) A single tag, used on "people on cellphone" viewed from any and all angles. This would include people facing sideways, frontwards, in between, etc, and ideally this single tag is trained enough to recognize multiple angles

B) Several tags, each specific to the angle that the person is viewed. For example, separate tags for: Facing Forwards, Facing Left, Facing Right, Arms Up, etc with each tag specializing in only subjects in that specific pose.

We felt that the drawback to having several tags would be that some of the tags have too few instances, and may not work as accurately as desired. And thus, missed.
But we are wondering if combining all the various angles into one tag cause it to be slightly less accurate in some way.

Again, any and all thought are appreciated! Thank you!

Ryan 21 Reputation points

2023-01-05T23:23:00+00:00

Hi @Ramr-msft , thank you for the time and help so far. I wanted to ask a couple more questions, and was hoping it was OK to just keep going in this thread. If it helps, I can re-post this as a brand new question if you would rather me do it that way.

1) I'm wondering what impact (if any) the size of the objects will have during the training and the AI recognition process. For example, we want to recognize people on cellphones in a fairly distant crowd. All of the recognition will be from the same somewhat far distance if that makes sense. (The subjects we are trying to recognize will always be roughly the same size in the images due to a fixed distance from the camera.)

So when we use images to train, or quick test, or ultimately run through the final iteration....the "person holding a cellphone" will always be roughly an inch tall on my monitor. When we train to recognize this scenario (specifically, little 1-inch people with phones here and there), does it help to train with a closeup picture of a person on a phone, that's maybe 6-8-10 inches tall on my screen? Should we try to train it to recognize the objects regardless of size? Or would it be better to try and train it with appropriately sized objects? Any thoughts you have are welcome and appreciated, thank you.
Ryan 21 Reputation points

2023-01-05T23:30:50.07+00:00

2) To increase recognition accuracy when recognizing "a person on a cellphone", which one of the following scenarios would be more accurate:

A) A single tag, used on "people on cellphone" viewed from any and all angles. This would include people facing sideways, frontwards, in between, etc, and ideally this single tag is trained enough to recognize multiple angles

B) Several tags, each specific to the angle that the person is viewed. For example, separate tags for: Facing Forwards, Facing Left, Facing Right, Arms Up, etc with each tag specializing in only subjects in that specific pose.

We felt that the drawback to having several tags would be that some of the tags have too few instances, and may not work as accurately as desired. And thus, missed.
But we are wondering if combining all the various angles into one tag cause it to be slightly less accurate in some way.

Again, any and all thought are appreciated! Thank you!

Answer 1

Ramr-msft 10,986

@Ryan Thanks for the question. There is no difference in the quality of the training data between the two flows you described. The main difference between the two flows is the order in which you perform the steps.

In the first flow, you first upload the images to the "Untagged" folder and then tag them. In the second flow, you first use the "Quick Test" feature to preview the results and then add/edit/correct the tags for the image.

Both flows will result in the same end result, which is that the images will be moved to the "Tagged" folder and will be used to train your object recognition model. It does not matter how the images end up in the "Trained" folder, as long as they are properly tagged and have been used to train the model.

In general, the most important thing is to make sure that your images are properly tagged and that you have a sufficient number of images for each object or class that you want to recognize. This will help ensure that your object recognition model is accurate and performs well.

Ryan 21 Reputation points

2022-12-30T21:41:53.81+00:00

I greatly appreciate the super fast response! You are saving me tremendous time by helping me find out which of our flows are simply unnecessary or even incorrect.

----
(1) Following up on your answer from my previous post. (You basically already answered this question by stating that there is no difference in the way an image ends up in the tagged folder, thank you.) I wanted confirm that there is no type of "Negative Training" or "Corrective Training" going on after correcting and tagging a Quick Test. A theory had been floated around here that if you Quick Test and correct the AI's suggested results, it learns even more from that "Corrective Step" of NOT accepting the AI tags, and then allowing that image to get to the Tagged bin. Does this exist?
Ryan 21 Reputation points

2022-12-30T21:42:36.347+00:00

(2) This question is regarding the images left in the Predictions tab after a quick test. These would be the images that the user didn't tag, and left there to be able to view results of that iteration.

Are these images in the Predictions tab cut off from everything else, or would they ever be referenced by the AI for training....or anything at all? My theory is that they can be safely left in the predictions tab without affecting current Tagged Images, without affecting Future Iteration Training, and without being pulled into "Tagged Bin" or "Untagged Bin" without the user directly doing so by editing within the Predictions tab.

Is this correct? Can my Quick Tested results images in the Predictions tab sit safely there? No need to remove them or worry that they are affecting future training?

You really are a huge help!
Ramr-msft 10,986 Reputation points

2023-01-02T11:29:42.353+00:00

1) No, correcting the tags on a Quick Test image will not have any special effect on the training process. When you correct the tags on a Quick Test image, you are simply adding or modifying the tags for that specific image. This will not affect the training process in any special way.

The Custom Vision service uses all of the images in the Trained folder, regardless of how they got there, to train the model. The tags on these images are used to teach the model what to look for in the images. The model will not pay special attention to images that were corrected during Quick Test or images that were added directly to the Trained folder.
Ramr-msft 10,986 Reputation points

2023-01-02T11:32:13.597+00:00

2) Yes, you are correct! Images in the Predictions tab are not used for training and will not affect the training process in any way. They are simply a preview of how the model is currently performing on those images. You can leave them in the Predictions tab without affecting the current or future training of the model.

The Custom Vision service only uses the images in the Trained folder for training. These images must be tagged with the objects you want the model to recognize. The tags on these images are used to teach the model what to look for in the images. The model will not pay attention to any images that are left in the Predictions tab.
Ryan 21 Reputation points

2023-01-05T17:17:19.487+00:00

Thanks for all the support! Your answers were exactly what I was looking for....extremely fast as well. Much, much appreciated!!

Custom Vision workflow question regarding Tagging and Training during object detection

0 additional answers

Activity