Azure AI Studio Index creation fails with "404 resource not found error"

Question

I am engaged in Azure AI Studio since the first week of December to configure a simple RAG service. It was pretty straighforward and i successfully created everything from AI studio. However, I encountered a challenge recently where the creation of an index consistently fails at the "LLM - Generate Embeddings" component within the pipeline.

Upon investigation, I identified that the root cause of this issue stems from a recent update of "LLM - Generate Embeddings", transitioning the version from 0.0.37 to 0.0.41. As a consequence of this update, the index creation job fails. To resolve this, I have to clone and revert the version of the "LLM - Generate Embeddings" component back to 0.0.37 for a successful execution of the index creation.

I am reaching out to inquire if there is a known resolution or fix for this specific challenge.

Here is the error log that appears when the LLM Generate Embedding component fails.

Execution failed. User process 'python' exited with status code 1. Please check log file 'user_logs/std_log.txt' for error details. Error: Traceback (most recent call last):
  File "/azureml-envs/rag-embeddings/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/azureml-envs/rag-embeddings/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/tasks/embed.py", line 506, in <module>
    main_wrapper(args, logger)
  File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/tasks/embed.py", line 473, in main_wrapper
    main(args, logger, activity_logger)
  File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/tasks/embed.py", line 456, in main
    create_embeddings(
  File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/tasks/embed.py", line 345, in create_embeddings
    _check_workers(embedder_futures, activity_logger)
  File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/tasks/embed.py", line 198, in _check_workers
    future.get(timeout=1)
  File "/azureml-envs/rag-embeddings/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
RuntimeError: Failed to embed 16 documents after 0s and 9 retries. Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

Answer 1

I'm not aware of this post recently. Since the issue posted here was found 8 months ago, probably the bug (if it was one ) had been fixed? Please try this scenario. If you are resubmitting a failed pipeline, it's very possible that it will prompt a message like there is newer version available. Please do upgrade the components in your pipeline.

The 404 error means the pipeline cannot find or access the embedding model with given AOAI connection. With that AOAI connection, there are few things you might want to check.

1. Is the endpoint, and API key set correctly. The endpoint should look like "https://<aoai_service_name>.openai.azure.com/"
2. If the connection is using Microsoft Entra ID as Authentication method instead of API key. please also make sure the user (who is creating the index) is assigned with "Cognitive Services OpenAI Contributor" role on the AOAI service.
3. There is embedding model deployed.