Custom Vision docker prediction model

Question

When I have a trained Custom Vision model that I have exported as a docker container and I run it on my device completely offline , I assume that the capability limits of my model endpoint in my docker container image still apply in line with the tier that I am using for Custom Vision? So for the free tier version of CV it will still be limited to 10 000 predictions per month or is that not quite true? Are there docker commands I can use to check on my prediction endpoint running in docker to check my remaining predictions quota?

Answer 1

@Batsirai Tambo Thanks for the question.
• Can we deploy the model and perform the scoring on device without needing internet connectivity all the time?
Yes for our compact models you can export them with no requirement for connectivity after you have the model on the device.
• Does the pricing change if it is deployed on-prem? How does Azure count the number of transactions if we are operating offline?
For compact models exported to the edge we DO NOT charge for prediction.

Export only makes sense if you want to have offline inference capabilities (no/low internet access).
You do not have to export it at all, you will get better results from calling the managed Custom Vision service (if you can) because the models are not using compact Neural Network architectures.

Here is an example iOS Xamarin app that we built for the AI Girls Hackfest in NYC that simply uses Custom Vision directly.
it’s basically a REST API call to the inference endpoint.