Use Baseten on Microsoft Foundry

Through integration with Baseten AI, Microsoft Foundry customers can:

  • Build AI applications with leading open models served by Baseten. Deploy directly from the Foundry model catalog as a Microsoft first-party offering.
  • Apply your Baseten model usage toward your Microsoft Azure Consumption Commitment (MACC), with usage billed through Azure.

You can use these capabilities in your Foundry project, with Azure governance, access controls, and project management built in.

Prerequisites

  • An Azure subscription. If you don't have one, create a free account.

  • A Foundry resource and project. You can also create more projects in an existing resource.

  • An identity with the Owner or Contributor role at subscription scope to enable the preview feature.

  • The Foundry Owner role on the Foundry project to deploy models. For more information, see Permissions for each built-in role.

    Important

    The Foundry RBAC roles were recently renamed. Foundry User, Foundry Owner, Foundry Account Owner, and Foundry Project Manager were previously named Azure AI User, Azure AI Owner, Azure AI Account Owner, and Azure AI Project Manager. You might still see the previous names in some places while the rename rolls out. The role IDs and core permissions are unchanged by the rename.

Check region availability

For supported deployment types and regions, see Region availability in the Baseten model catalog reference.

Enable Baseten on Foundry

An administrator must enable Baseten on Foundry for your Azure subscription.

Important

Baseten on Foundry is a non-Microsoft product. Baseten services are operated outside Microsoft facilities. Customer data is shared with Baseten and processed under the Baseten Data Protection Addendum, Terms and Conditions, and Privacy Policy. Review Data privacy before you enable the service.

  1. Sign in to the Azure portal.
  2. Search for and select Subscriptions.
  3. Select the subscription name.
  4. Under Settings, select Preview features.
  5. Search for and select Baseten.EnableDeploy.
  6. Review the Description, terms, and data privacy information.
  7. If you don't agree, select Close and don't continue. Otherwise, select Register.
  8. Select OK. The Preview features page refreshes and displays the feature state. Registration can take up to 30 minutes.

Tip

Refresh the page and confirm that the state for Baseten on Foundry is Registered.

For more information about preview feature registration, see Set up preview features in an Azure subscription.

Deploy a Baseten model

After you enable the feature, deploy a Baseten model from the Foundry model catalog to get a live Chat Completions endpoint.

  1. On the Foundry portal home page, select Catalog.
  2. In the catalog navigation, select Models.
  3. Under Collections, select Baseten on Foundry.
  4. Select a model to open its details page.
  5. Select Deploy. For deployment guidance, see Deploy Foundry Models in the portal.
  6. Configure these deployment settings:
    • Deployment name: Keep the default name or enter a custom name.
    • Deployment type: Under Pay-per-token, select Data Zone Standard or Global Standard. For more information, see Deployment types.
    • Tokens per minute rate limit: Enter a custom limit to manage cost and usage. For more information, see Manage quotas and rate limits.
    • Guardrails: Select DefaultV2 or Default. Unless you specify another option, Microsoft.DefaultV2 is the default. For more information, see Default guardrails.
  7. Select Deploy. Deployment can take up to 30 minutes.
  8. After deployment finishes, get the endpoint and key from the deployment details. Use the Playground to test the deployment.

Tip

To verify the deployment, go to your project's Deployments page and confirm the deployment Status shows Succeeded.

Use the Chat Completions API

Baseten catalog models use the OpenAI v1 Chat Completions API format with a Foundry endpoint and authentication. In the deployment details, get the endpoint and credentials for your deployment.

Before you run the Python sample:

  • Install the openai package.
  • Set FOUNDRY_BASE_URL to a URL in the format https://<resource-name>.services.ai.azure.com/openai/v1/.
  • Set FOUNDRY_API_KEY to your deployment key. Never store an actual key in source code.
  • Set FOUNDRY_DEPLOYMENT_NAME to your deployment name.
import os

from openai import OpenAI

client = OpenAI(
    base_url=os.environ["FOUNDRY_BASE_URL"],
    api_key=os.environ["FOUNDRY_API_KEY"],
)

response = client.chat.completions.create(
    model=os.environ["FOUNDRY_DEPLOYMENT_NAME"],
    messages=[
        {
            "role": "user",
            "content": "Explain retrieval-augmented generation briefly.",
        }
    ],
)
print(response.choices[0].message.content)

Reference: Azure OpenAI API version lifecycle

Manage quotas and rate limits

For pay-per-token deployments, Global Standard and Data Zone Standard use separate quota pools. The default enterprise quota is 10 million tokens per minute (TPM) for Global Standard and 5 million TPM for each data zone, per subscription. Currently, only the US data zone is available.

Each pool is shared across all Baseten models. Allocate quota to individual deployments from the shared pool.

Baseten applies adaptive rate limits within your quota. Effective limits can be lower than the full quota and grow automatically with usage. Ramp up traffic gradually, and use exponential backoff when retrying HTTP 429 responses.

All customers can request a quota increase.

Available catalog models

For the supported catalog models, deployment types, regions, and pay-per-token retirement dates, see Baseten model catalog.

All catalog models support the OpenAI/v1 API for chat completions.

Review data privacy

Baseten services operate outside Microsoft facilities. By selecting Deploy, you agree on behalf of your organization to be bound by and use the Baseten services only according to the Baseten Data Protection Addendum, Terms and Conditions, and Privacy Policy.

Review the Baseten Trust Center and the privacy and compliance FAQ before deployment.

Review the transparency note

Baseten on Foundry provides third-party, open-weight models through the Foundry platform. Microsoft doesn't develop, train, fine-tune, or evaluate the safety, security, or Responsible AI characteristics of these models. Microsoft makes no representations about their behavior, performance, or risk.

You're solely responsible for assessing each model's suitability, including its safety, compliance, and Responsible AI characteristics, before using it in production or customer-facing scenarios. Foundry provides tools to run your own risk and safety evaluations. For more information, see the safety evaluations transparency note.

Frequently asked questions

Is Baseten on Foundry available in Azure Government?

No. Baseten on Foundry isn't available in Azure Government.

How do I request more quota?

Use the quota request form to request more quota for Baseten on Foundry models.

Can I use existing Baseten accounts or deployments?

No. Create new Foundry deployments. Contact your Baseten account team for help shifting consumption to Azure.

How am I billed?

Azure bills Baseten model usage on a pay-per-token basis.

How do I disable Baseten for my subscription?

You can disable Baseten at the Azure subscription level. Follow the steps to unregister preview features in your Azure subscription.

Which inference API can I use?

Use the Chat Completions API. See Use the Chat Completions API.

Troubleshoot deployment issues

Use the following table to resolve common deployment issues.

Issue Resolution
Registration remains Registering Registration can take up to 30 minutes. Refresh the page. If it remains Registering for more than 30 minutes, unregister and register the feature again.
Baseten models don't appear in the catalog Confirm that Baseten.EnableDeploy is Registered and that you're using a supported region.
A quota error prevents deployment Use the quota increase request form.
A Forbidden error prevents deployment Confirm that your identity has the Foundry Owner role on the Foundry project. Subscription roles alone aren't sufficient.

For other questions, see Frequently asked questions.