Transitioning from Azure AI Image Analysis to Florence 2.0 for Background Removal

Question

Transitioning from Azure AI Image Analysis to Florence 2.0 for Background Removal

Julian Korevaar 15

Hi everyone,

I recently received an email from Azure informing me that on January 10, 2025, the Azure AI Image Analysis 4.0 Segment API and its background removal service will be retired. I’ve been using this service to remove backgrounds from uploaded images, and I have a few questions regarding the transition.

The email mentions that to avoid service disruptions, we should switch to Florence 2.0 by the specified date. As someone who’s relatively new to computer vision and AI, the Segment API was a perfect solution for my needs.

It seems like Florence 2.0 offers a "Region to Segmentation" feature, but I’m not sure what steps I need to take to achieve the same outcome — specifically, creating a transparent image with the background removed.

Could anyone guide me on how to use Florence 2.0 for background removal? Any help or advice on this transition would be greatly appreciated!

Thank you,

Julian

SRILAKSHMI C 14,140 Reputation points Microsoft External Staff Moderator

2024-09-27T09:00:07.31+00:00

Hello Julian Korevaar, Greetings!

We are reaching out to the internal team to get more information related to your query and will get back to you as soon as we have an update.

Thank you!
SRILAKSHMI C 14,140 Reputation points Microsoft External Staff Moderator

2024-09-27T13:52:52.2733333+00:00
Hello Julian Korevaar,

Thank you for reaching out to the Microsoft Q&A forum.

To continue with background removal after the retirement of the Azure AI Image Analysis 4.0 Segment API, you can switch to using Florence 2.0. The Florence 2.0 is an advanced foundation model trained on billions of text-image pairs and is integrated into Azure Cognitive Services for Vision. It includes a "Region to Segmentation" feature that can achieve similar outcomes as the Segment API.

To use Florence 2.0 for background removal, you can process your images through Azure Computer Vision. Upload your images, which will segment the image into foreground and background. Once the segmentation is done, you’ll get a mask highlighting the foreground object.

Florence 2.0 is a cost-effective, production-ready service that provides enhanced capabilities, allowing you to continue your computer vision tasks seamlessly.

For more info, please refer to:

Announcing a renaissance in computer vision AI with Microsoft

Background removal - Image Analysis - Azure AI services

GitHub - Azure-Samples/azure-ai-vision-sdk: SDK for Microsoft's Azure AI Vision

I hope this answer will helpful. Thank you!
Julian Korevaar 15 Reputation points

2024-09-27T18:52:25.3033333+00:00
Hi @Anonymous ,

Thanks for the response.

I have one follow-up question regarding this Florence 2.0 model that can be used. Right now I am doing an API call to the segment API and send the image in the url of the body. After that call i get the returned image without the background.

~url~/computervision/imageanalysis:segment?api-version=2023-02-01-preview&mode=backgroundRemoval

My question is, what approach would you recomment to also use the Florence 2.0 in an automated way while using an API, and is that even possible?

Hope to hear from you soon!

Kind regards,

Julian
SRILAKSHMI C 14,140 Reputation points Microsoft External Staff Moderator

2024-10-01T09:40:32.7066667+00:00

Hello Julian Korevaar,

You can automate background removal with Florence 2.0 using Azure Computer Vision's "Region to Segmentation" feature. Send your image via an API call similar to the Segment API, get the segmentation mask, and apply it to the original image to remove the background.
Tabareh 25 Reputation points

2024-10-17T10:50:23.98+00:00

Hello!

We have a very similar situation with a service in production and we're interested into knowing the way forward. I do not find anything in the documentation that explains a replacement feature in Azure Portal. All the links are going to Florence segmentation in hugging face.

Could you please elaborate on the solution with a sample url/documentation for api call for the new way of working? The links you provided before leads to the page with warning for retirement on January 2025.

Thanks in advance.
Soeth, Niels 10 Reputation points

2024-11-05T05:28:51.7766667+00:00

Any updates on this? I can't figure out how to remove the background using Florence. Segmentation is not the same and cannot be used for removing the background from images.
Tabareh 25 Reputation points

2024-11-11T08:44:11.0366667+00:00

Hello again!

The time is running out and our production systems are in danger. Could someone provide a guide or documentation for the api that is mentioned? Or any other way to deploy Florence on Azure and use it that way?

I would like to express that this decommission timeline is not realistic with the level of support that we get to do the transition. Could it be postponed somehow?

Kind regards,

Adel
SRILAKSHMI C 14,140 Reputation points Microsoft External Staff Moderator

2024-11-11T09:19:44.59+00:00

Hi Tabareh,

I would recommend creating a new thread on the same forum with as many details about your issue as possible. That would make sure that your issue has better visibility in the community.
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

4 answers

Your answer

SRILAKSHMI C 14,140 Reputation points Microsoft External Staff Moderator

2024-09-27T09:00:07.31+00:00

Hello Julian Korevaar, Greetings!

We are reaching out to the internal team to get more information related to your query and will get back to you as soon as we have an update.

Thank you!
SRILAKSHMI C 14,140 Reputation points Microsoft External Staff Moderator

2024-09-27T13:52:52.2733333+00:00

Hello Julian Korevaar,

Thank you for reaching out to the Microsoft Q&A forum.

To continue with background removal after the retirement of the Azure AI Image Analysis 4.0 Segment API, you can switch to using Florence 2.0. The Florence 2.0 is an advanced foundation model trained on billions of text-image pairs and is integrated into Azure Cognitive Services for Vision. It includes a "Region to Segmentation" feature that can achieve similar outcomes as the Segment API.

To use Florence 2.0 for background removal, you can process your images through Azure Computer Vision. Upload your images, which will segment the image into foreground and background. Once the segmentation is done, you’ll get a mask highlighting the foreground object.

Florence 2.0 is a cost-effective, production-ready service that provides enhanced capabilities, allowing you to continue your computer vision tasks seamlessly.

For more info, please refer to:

Announcing a renaissance in computer vision AI with Microsoft

Background removal - Image Analysis - Azure AI services

GitHub - Azure-Samples/azure-ai-vision-sdk: SDK for Microsoft's Azure AI Vision

I hope this answer will helpful. Thank you!
Julian Korevaar 15 Reputation points

2024-09-27T18:52:25.3033333+00:00

Hi @Anonymous ,

Thanks for the response.

I have one follow-up question regarding this Florence 2.0 model that can be used. Right now I am doing an API call to the segment API and send the image in the url of the body. After that call i get the returned image without the background.

~url~/computervision/imageanalysis:segment?api-version=2023-02-01-preview&mode=backgroundRemoval

My question is, what approach would you recomment to also use the Florence 2.0 in an automated way while using an API, and is that even possible?

Hope to hear from you soon!

Kind regards,

Julian
SRILAKSHMI C 14,140 Reputation points Microsoft External Staff Moderator

2024-10-01T09:40:32.7066667+00:00

Hello Julian Korevaar,

You can automate background removal with Florence 2.0 using Azure Computer Vision's "Region to Segmentation" feature. Send your image via an API call similar to the Segment API, get the segmentation mask, and apply it to the original image to remove the background.
Tabareh 25 Reputation points

2024-10-17T10:50:23.98+00:00

Hello!

We have a very similar situation with a service in production and we're interested into knowing the way forward. I do not find anything in the documentation that explains a replacement feature in Azure Portal. All the links are going to Florence segmentation in hugging face.

Could you please elaborate on the solution with a sample url/documentation for api call for the new way of working? The links you provided before leads to the page with warning for retirement on January 2025.

Thanks in advance.
Soeth, Niels 10 Reputation points

2024-11-05T05:28:51.7766667+00:00

Any updates on this? I can't figure out how to remove the background using Florence. Segmentation is not the same and cannot be used for removing the background from images.
Tabareh 25 Reputation points

2024-11-11T08:44:11.0366667+00:00

Hello again!

The time is running out and our production systems are in danger. Could someone provide a guide or documentation for the api that is mentioned? Or any other way to deploy Florence on Azure and use it that way?

I would like to express that this decommission timeline is not realistic with the level of support that we get to do the transition. Could it be postponed somehow?

Kind regards,

Adel
SRILAKSHMI C 14,140 Reputation points Microsoft External Staff Moderator

2024-11-11T09:19:44.59+00:00

Hi Tabareh,

I would recommend creating a new thread on the same forum with as many details about your issue as possible. That would make sure that your issue has better visibility in the community.
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

Answer 1

With Azure retiring the Image Analysis 4.0 Segment API in early 2025, Florence 2.0’s “Region to Segmentation” within Azure Computer Vision is the recommended substitute.

If you're building a pipeline, you'll want to:

Call the new Computer Vision /analyze endpoint with the region-to-segmentation parameter.

Use the returned mask to composite a transparent-background image.

Automate it via your existing codebase—Florence 2.0 supports standard API access just like the old Segment API

We’ve been experimenting with similar setups over at just build things ai image analyzer, working on end-to-end image-background removal using Florence. Happy to share sample Node.js or Python snippets if you're interested!

Answer 2

Alex Hanneman 0

We experimented with Florence 2 and is not even close to as good as Florence 1 (Azure Computer Vision) for background removal of human portraits. Additionally, the AI model architecture outputs polygons which requires a high amount of looping for post processing the image. Other AI models output prediction of pixels which allows for a much faster image post processing.

Alex Hanneman 0 Reputation points

2024-12-13T21:32:36.7466667+00:00

Alternatively there are plenty of APIs available you can pay for.

Our solution ended up using an open source ONNX model and the Microsoft.ML.OnnxRuntime nuget package
rocco@dobles 0 Reputation points

2024-12-19T16:38:05.7933333+00:00

uff this sounds alarming. I need to replace this api within 20 days as well, and have no idea what to go for. What's the ONNX model you ended up going for?
rocco@dobles 0 Reputation points

2024-12-19T16:38:56.9566667+00:00

what's the open source ONNX model you ended up using? Does it produce comparable results?

Answer 3

Anonymous

srilakshmic - what do you mean by "To use Florence 2.0 for background removal, you can process your images through Azure Computer Vision." ?

I was under the impression the only way forward is the whole semantic kernel w/ huggingface route and not the (very convenient) background removal as a service, we have (had?) with Computer Vision.

Can you clarify please?

Keith Nicholson 36 Reputation points

2024-12-03T04:30:32.79+00:00

I agree with Neil, the comment to use the item being deprecated seems like an infinite loop. Will Microsoft provide good examples? How does pricing work? I'm in an enterprise and no way will I get pricing approval outside of our EA this late in 4th quarter.
Alex Hanneman 0 Reputation points

2024-12-13T18:23:57.6366667+00:00

Using Azure ML you can deploy an AI model and then define an endpoint for inference https://learn.microsoft.com/en-us/azure/machine-learning/how-to-deploy-online-endpoints?view=azureml-api-2&tabs=cli

Florence 2 is open source so you can download it and then upload it Azure ML.
rocco@dobles 0 Reputation points

2024-12-19T16:37:04.7+00:00

Yes. I'm also clueless. Please provide clarifications on the best way forward.

Answer 4

Tabareh 25

@Julian Korevaar Have you found any solution/workaround for that?

Keith Nicholson 36 Reputation points

2024-11-29T03:14:18.6866667+00:00

This is troublesome as we have a similar issue. If Microsoft is going to reference the API, it would be nice to have some sample code in Github.
Julian Korevaar 15 Reputation points

2024-12-17T19:20:08.19+00:00

@Tabareh Sorry for the late response to your question. Since we are using a Node.js backend structure we could fall back on using the @imgly/background-removal-node package. However, it needs to be noted that this can not be compared to the quality of the Azure Computer Vision. It does the job though if quality is not your main priority.

Share via

Transitioning from Azure AI Image Analysis to Florence 2.0 for Background Removal

4 answers

Your answer