Edit

Remove the background from images

  • Article

This article demonstrates how to call the Image Analysis 4.0 API to segment an image. It also shows you how to parse the returned information.

Prerequisites

This guide assumes you have successfully followed the steps mentioned in the quickstart page. This means:

  • You have created a Vision resource and obtained a key and endpoint URL.
  • If you're using the client SDK, you have the appropriate SDK package installed and you have a running quickstart application. You modify this quickstart application based on code examples here.
  • If you're using 4.0 REST API calls directly, you have successfully made a curl.exe call to the service (or used an alternative tool). You modify the curl.exe call based on the examples here.

The quickstart shows you how to extract visual features from an image, however, the concepts are similar to background removal. Therefore you benefit from starting from the quickstart and making modifications.

Important

Background removal is only available in the following Azure regions: East US, France Central, Korea Central, North Europe, Southeast Asia, West Europe, West US.

Authenticate against the service

To authenticate against the Image Analysis service, you need an Azure AI Vision key and endpoint URL.

Tip

Don't include the key directly in your code, and never post it publicly. See the Azure AI services security article for more authentication options like Azure Key Vault.

The SDK example assumes that you defined the environment variables VISION_KEY and VISION_ENDPOINT with your key and endpoint.

Start by creating a VisionServiceOptions object using one of the constructors. For example:

var serviceOptions = new VisionServiceOptions(
    Environment.GetEnvironmentVariable("VISION_ENDPOINT"),
    new AzureKeyCredential(Environment.GetEnvironmentVariable("VISION_KEY")));

Select the image to analyze

The code in this guide uses remote images referenced by URL. You may want to try different images on your own to see the full capability of the Image Analysis features.

Create a new VisionSource object from the URL of the image you want to analyze, using the static constructor VisionSource.FromUrl.

using var imageSource = VisionSource.FromUrl(
    new Uri("https://learn.microsoft.com/azure/ai-services/computer-vision/media/quickstarts/presentation.png"));

VisionSource implements IDisposable, therefore create the object with a using statement or explicitly call Dispose method after analysis completes.

Tip

You can also analyze a local image by passing in the full-path image file name (see VisionSource.FromFile), or by copying the image into the SDK's input buffer (see VisionSource.FromImageSourceBuffer). For more details, see Call the Analyze API.

Select a mode

Create a new ImageAnalysisOptions object and set the property SegmentationMode. This property must be set if you want to do segmentation. See ImageSegmentationMode for supported values.

var analysisOptions = new ImageAnalysisOptions()
{
    SegmentationMode = ImageSegmentationMode.BackgroundRemoval
};

Get results from the service

This section shows you how to make the API call and parse the results.

The following code calls the Image Analysis API and saves the resulting segmented image to a file named output.png. It also displays some metadata about the segmented image.

using var analyzer = new ImageAnalyzer(serviceOptions, imageSource, analysisOptions);

var result = analyzer.Analyze();

if (result.Reason == ImageAnalysisResultReason.Analyzed)
{
    using var segmentationResult = result.SegmentationResult;

    var imageBuffer = segmentationResult.ImageBuffer;
    Console.WriteLine($" Segmentation result:");
    Console.WriteLine($"   Output image buffer size (bytes) = {imageBuffer.Length}");
    Console.WriteLine($"   Output image height = {segmentationResult.ImageHeight}");
    Console.WriteLine($"   Output image width = {segmentationResult.ImageWidth}");

    string outputImageFile = "output.png";
    using (var fs = new FileStream(outputImageFile, FileMode.Create))
    {
        fs.Write(imageBuffer.Span);
    }
    Console.WriteLine($"   File {outputImageFile} written to disk");
}
else
{
    var errorDetails = ImageAnalysisErrorDetails.FromResult(result);
    Console.WriteLine(" Analysis failed.");
    Console.WriteLine($"   Error reason : {errorDetails.Reason}");
    Console.WriteLine($"   Error code : {errorDetails.ErrorCode}");
    Console.WriteLine($"   Error message: {errorDetails.Message}");
    Console.WriteLine(" Did you set the computer vision endpoint and key?");
}

SegmentationResult implements IDisposable, therefore create the object with a using statement or explicitly call Dispose method after analysis completes.

As an example, assume background removal is run on the following image:

Photo of a city near water.

On a successful background removal call, The following four-channel PNG image is the response for the backgroundRemoval mode:

Photo of a city near water; sky is transparent.

The following one-channel PNG image is the response for the foregroundMatting mode:

Alpha matte of a city skyline.

The API returns an image the same size as the original for the foregroundMatting mode, but at most 16 megapixels (preserving image aspect ratio) for the backgroundRemoval mode.

Error codes

The sample code for getting analysis results shows how to handle errors and get the ImageAnalysisErrorDetails object that contains the error information. The error information includes:

  • Error reason. See enum ImageAnalysisErrorReason.
  • Error code and error message. Select the REST API tab to see a list of some common error codes and messages.

In addition to those errors, the SDK has a few other error messages, including:

  • Missing Image Analysis options: You must set at least one visual feature (or model name) for the 'analyze' operation. Or set segmentation mode for the 'segment' operation
  • Invalid combination of Image Analysis options: You cannot set both visual features (or model name), and segmentation mode

Make sure the ImageAnalysisOptions object is set correctly to fix these errors.

To help resolve issues, look at the Image Analysis Samples repository and run the closest sample to your scenario. Search the GitHub issues to see if your issue was already address. If not, create a new.

Tip

While working with Azure AI Vision, you might encounter transient failures caused by rate limits enforced by the service, or other transient problems like network outages. For information about handling these types of failures, see Retry pattern in the Cloud Design Patterns guide, and the related Circuit Breaker pattern.

Next steps

Background removal concepts