Unable to Enable Vector Search in Azure OpenAI Chat Playground for Azure AI Search Data Source

Question

When I am trying to create a chat playground I can't enable vector search for my "Azure AI Search" data source. The index has embedding included and I have embedding in my deployment.

The steps I have taken to get here.

1. Go to Azure OpenAI Service
2. Added two deployments
   1. gpt-4o-mini
   2. text-embedding-3-large
3. Tried to use chat playground
   1. Selected my deployment
      1. gpt-4o-mini (version: 2024-07-18)
   2. Tried To Add Data Source With Vector Search Enabled
      1. clicked "Add a data Source"
      2. Selected "Azure AI Search" for my data source
      3. Selected my subscription
      4. Selected my Azure AI Search Service
      5. Selected my Azure AI Search Index
      6. At this point I would like to check the "Add vector search ..." checkbox but it is disabled

The checkbox "Add vector search..." is disabled.

Here is the index's json if that helps:

{
  "name": "knowledge-index",
  "fields": [
    {
      "name": "knowledge_id",
      "type": "Edm.String",
      "key": true,
      "retrievable": true,
      "stored": true,
      "searchable": false,
      "filterable": true,
      "sortable": false,
      "facetable": false,
      "synonymMaps": []
    },
    {
      "name": "contact_dt",
      "type": "Edm.String",
      "key": false,
      "retrievable": true,
      "stored": true,
      "searchable": false,
      "filterable": true,
      "sortable": false,
      "facetable": false,
      "synonymMaps": []
    },
    {
      "name": "reference_num",
      "type": "Edm.String",
      "key": false,
      "retrievable": true,
      "stored": true,
      "searchable": true,
      "filterable": true,
      "sortable": true,
      "facetable": true,
      "synonymMaps": []
    },
    {
      "name": "vector",
      "type": "Collection(Edm.Single)",
      "key": false,
      "retrievable": true,
      "stored": true,
      "searchable": true,
      "filterable": false,
      "sortable": false,
      "facetable": false,
      "synonymMaps": [],
      "dimensions": 3072,
      "vectorSearchProfile": "myHnswProfile"
    }
  ],
  "scoringProfiles": [],
  "corsOptions": {
    "allowedOrigins": [
      "*"
    ],
    "maxAgeInSeconds": 300
  },
  "suggesters": [],
  "analyzers": [],
  "tokenizers": [],
  "tokenFilters": [],
  "charFilters": [],
  "normalizers": [],
  "similarity": {
    "@odata.type": "#Microsoft.Azure.Search.BM25Similarity"
  },
  "semantic": {
    "configurations": [
      {
        "name": "mySemanticConfig",
        "prioritizedFields": {
          "titleField": {
            "fieldName": "serial_num"
          },
          "prioritizedContentFields": [
            {
              "fieldName": "text_serial_model"
            }
          ],
          "prioritizedKeywordsFields": []
        }
      }
    ]
  },
  "vectorSearch": {
    "algorithms": [
      {
        "name": "myHnsw",
        "kind": "hnsw",
        "hnswParameters": {
          "m": 4,
          "efConstruction": 400,
          "efSearch": 500,
          "metric": "cosine"
        }
      },
      {
        "name": "myExhaustiveKnn",
        "kind": "exhaustiveKnn",
        "exhaustiveKnnParameters": {
          "metric": "cosine"
        }
      }
    ],
    "profiles": [
      {
        "name": "myHnswProfile",
        "algorithm": "myHnsw",
        "vectorizer": "vectorizer-1732168287035"
      },
      {
        "name": "myExhaustiveKnnProfile",
        "algorithm": "myExhaustiveKnn",
        "vectorizer": "vectorizer-1732168287035"
      },
      {
        "name": "vector-profile-1732168273201",
        "algorithm": "myHnsw",
        "vectorizer": "vectorizer-1732168287035"
      }
    ],
    "vectorizers": [
      {
        "name": "vectorizer-1732168287035",
        "kind": "azureOpenAI",
        "azureOpenAIParameters": {
          "resourceUri": "https://oai-east.openai.azure.com",
          "deploymentId": "text-embedding-3-large",
          "apiKey": "<redacted>",
          "modelName": "text-embedding-3-large"
        }
      }
    ],
    "compressions": []
  },
  "@odata.etag": "\"0x8DD09F1043B4F27\""
}

I would like to use both vector and semantic. If I figure out how to enable vector search will the request converted to vectors and then sent to the model?

My desired result is that I will be able to enable vector search against my index in the chat playground.

Answer 1

@Jeremy Vetter I have tried to add your JSON as an index to my search resource and while adding I removed the semantic section of the JSON and used it in my BYOD configuration of OpenAI model. See the screenshots below:

Vector Profiles:

No Semantic Config:

Now, using the same index in the BYOD screen, it enables the option to enable vector search.

In the Next screen you have the option enable Hybrid search (Vector + Semantic) choose this option and continue rest of the setup.

Where Hybrid ( vector + keyword) is:

See this page for details.

For your other question, If you enable vector search Azure OpenAI uses the selected embedding setting to vectorize the chunks. This is documented on this page. I hope this helps!!

Answer 2

Came here because I had the same problem.

This option is not available for the embedding deployment with model "text-embedding-3-large". As soon as you deploy the model "text-embedding-ada-002" in your OpenAI Service instance, the option to use Vector Search with your Azure AI Search Data Source is enabled.

This is a major drawback, as now the vector dimensions need to match and your Search Index embedding must also use the "text-embedding-ada-002" OpenAI model as vectorizer.

Answer 3

Jeeezzz, I was about to despair about this. Nothing of the above seemed to help. For my case, ultimately, the solution was enabling CORS. See this page: https://learn.microsoft.com/en-us/azure/ai-foundry/openai/concepts/use-your-data?tabs=ai-search%2Ccopilot#regional-availability-and-model-support

After having created the indexer, index and data source using the "Import and vectorize data" wizard, I had to navigate to the index's own settings and simply enable CORS. See screenshot:

EDIT: Gosh, I think I need to go to bed. The entire system is throwing randomly different error messages all the time. It looks like behind the scenes there is a mess going on.

400: The specified embedding deployment 'text-embedding-ada-002' in the request body does not exist.

After lots of trying out I figured that if I select text-embedding-ada-002 as a model then the indexer fails with a cryptic error message. BUT, if I deploy any other models like the text-embedding-3-small THEN during I cannot select this model from the dropdown list of available models, although I do have quota available for this model.

Which essentially means: this is a catch-22 - I cannot select the ADA model (because then the indexer fails), nor can I select the 3-small model (because that's not even available in Azure AI Foundry wizard).

Nice one, Microsoft!

EDIT 2: The insanity continues. It seems that now, after having wasted another hour or so, suddenly text-embedding-3-small now IS available in Azure AI Foundry wizard. I have no clue anymore what's going on.