Azure Cognitive Services Computer Vision SDK for Python
The Computer Vision service provides developers with access to advanced algorithms for processing images and returning information. Computer Vision algorithms analyze the content of an image in different ways, depending on the visual features you're interested in.
You can use Computer Vision in your application to:
- Analyze images for insight
- Extract text from images
- Generate thumbnails
Looking for more documentation?
If you need a Computer Vision API account, you can create one with this Azure CLI command:
RES_REGION=westeurope RES_GROUP=<resourcegroup-name> ACCT_NAME=<computervision-account-name> az cognitiveservices account create \ --resource-group $RES_GROUP \ --name $ACCT_NAME \ --location $RES_REGION \ --kind ComputerVision \ --sku S1 \ --yes
Configure a virtual environment (optional)
Although not required, you can keep your base system and Azure SDK environments isolated from one another if you use a virtual environment. Execute the following commands to configure and then enter a virtual environment with venv, such as
python3 -m venv cogsrv-vision-env source cogsrv-vision-env/bin/activate
Install the SDK
pip install azure-cognitiveservices-vision-computervision
Once you create your Computer Vision resource, you need its region, and one of its account keys to instantiate the client object.
Use these values when you create the instance of the ComputerVisionClient client object.
Use the Azure CLI snippet below to populate two environment variables with the Computer Vision account region and one of its keys (you can also find these values in the Azure portal). The snippet is formatted for the Bash shell.
RES_GROUP=<resourcegroup-name> ACCT_NAME=<computervision-account-name> export ACCOUNT_REGION=$(az cognitiveservices account show \ --resource-group $RES_GROUP \ --name $ACCT_NAME \ --query location \ --output tsv) export ACCOUNT_KEY=$(az cognitiveservices account keys list \ --resource-group $RES_GROUP \ --name $ACCT_NAME \ --query key1 \ --output tsv)
Once you've populated the
ACCOUNT_KEY environment variables, you can create the ComputerVisionClient client object.
from azure.cognitiveservices.vision.computervision import ComputerVisionClient from azure.cognitiveservices.vision.computervision.models import VisualFeatureTypes from msrest.authentication import CognitiveServicesCredentials import os region = os.environ['ACCOUNT_REGION'] key = os.environ['ACCOUNT_KEY'] credentials = CognitiveServicesCredentials(key) client = ComputerVisionClient( endpoint="https://" + region + ".api.cognitive.microsoft.com/", credentials=credentials )
Once you've initialized a ComputerVisionClient client object, you can:
- Analyze an image: You can analyze an image for certain features such as faces, colors, tags.
- Generate thumbnails: Create a custom JPEG image to use as a thumbnail of the original image.
- Get description of an image: Get a description of the image based on its subject domain.
For more information about this service, see What is Computer Vision?.
The following sections provide several code snippets covering some of the most common Computer Vision tasks, including:
- Analyze an image
- Get subject domain list
- Analyze an image by domain
- Get text description of an image
- Get handwritten text from image
- Generate thumbnail
Analyze an image
You can analyze an image for certain features with
analyze_image. Use the
visual_features property to set the types of analysis to perform on the image. Common values are
url = "https://upload.wikimedia.org/wikipedia/commons/thumb/1/12/Broadway_and_Times_Square_by_night.jpg/450px-Broadway_and_Times_Square_by_night.jpg" image_analysis = client.analyze_image(url,visual_features=[VisualFeatureTypes.tags]) for tag in image_analysis.tags: print(tag)
Get subject domain list
models = client.list_models() for x in models.models_property: print(x)
Analyze an image by domain
domain = "landmarks" url = "https://images.pexels.com/photos/338515/pexels-photo-338515.jpeg" language = "en" analysis = client.analyze_image_by_domain(domain, url, language) for landmark in analysis.result["landmarks"]: print(landmark["name"]) print(landmark["confidence"])
Get text description of an image
You can get a language-based text description of an image with
describe_image. Request several descriptions with the
max_description property if you are doing text analysis for keywords associated with the image. Examples of a text description for the following image include
a train crossing a bridge over a body of water,
a large bridge over a body of water, and
a train crossing a bridge over a large body of water.
domain = "landmarks" url = "http://www.public-domain-photos.com/free-stock-photos-4/travel/san-francisco/golden-gate-bridge-in-san-francisco.jpg" language = "en" max_descriptions = 3 analysis = client.describe_image(url, max_descriptions, language) for caption in analysis.captions: print(caption.text) print(caption.confidence)
Get text from image
You can get any handwritten or printed text from an image. This requires two calls to the SDK:
get_read_result. The call to read is asynchronous. In the results of the get_read_result call, you need to check if the first call completed with
OperationStatusCodes before extracting the text data. The results include the text as well as the bounding box coordinates for the text.
# import models from azure.cognitiveservices.vision.computervision.models import OperationStatusCodes url = "https://github.com/Azure-Samples/cognitive-services-python-sdk-samples/raw/master/samples/vision/images/make_things_happen.jpg" raw = True numberOfCharsInOperationId = 36 # SDK call rawHttpResponse = client.read(url, language="en", raw=True) # Get ID from returned headers operationLocation = rawHttpResponse.headers["Operation-Location"] idLocation = len(operationLocation) - numberOfCharsInOperationId operationId = operationLocation[idLocation:] # SDK call result = client.get_read_result(operationId) # Get data if result.status == OperationStatusCodes.succeeded: for line in result.analyze_result.read_results.lines: print(line.text) print(line.bounding_box)
You can generate a thumbnail (JPG) of an image with
generate_thumbnail. The thumbnail does not need to be in the same proportions as the original image.
This example uses the Pillow package to save the new thumbnail image locally.
from PIL import Image import io width = 50 height = 50 url = "http://www.public-domain-photos.com/free-stock-photos-4/travel/san-francisco/golden-gate-bridge-in-san-francisco.jpg" thumbnail = client.generate_thumbnail(width, height, url) for x in thumbnail: image = Image.open(io.BytesIO(x)) image.save('thumbnail.jpg')
When you interact with the ComputerVisionClient client object using the Python SDK, the
ComputerVisionErrorException class is used to return errors. Errors returned by the service correspond to the same HTTP status codes returned for REST API requests.
For example, if you try to analyze an image with an invalid key, a
401 error is returned. In the following snippet, the error is handled gracefully by catching the exception and displaying additional information about the error.
domain = "landmarks" url = "http://www.public-domain-photos.com/free-stock-photos-4/travel/san-francisco/golden-gate-bridge-in-san-francisco.jpg" language = "en" max_descriptions = 3 try: analysis = client.describe_image(url, max_descriptions, language) for caption in analysis.captions: print(caption.text) print(caption.confidence) except HTTPFailure as e: if e.status_code == 401: print("Error unauthorized. Make sure your key and region are correct.") else: raise
Handle transient errors with retries
While working with the ComputerVisionClient client, you might encounter transient failures caused by rate limits enforced by the service, or other transient problems like network outages. For information about handling these types of failures, see Retry pattern in the Cloud Design Patterns guide, and the related Circuit Breaker pattern.
More sample code
Several Computer Vision Python SDK samples are available to you in the SDK's GitHub repository. These samples provide example code for additional scenarios commonly encountered while working with Computer Vision:
For more extensive documentation on the Computer Vision service, see the Azure Computer Vision documentation on docs.microsoft.com.