I'm getting the error "No appropriate cache found for provided policy configuration. Policy execution will be skipped." . Semantic caching is not working for Azure OpenAI responses

Question

Context: I'm trying to set up an APIM service which has semantic caching enabled for GPT requests so that the response times for same/similar questions are lesser but the caching implementation is not working even though I am able to see the responses from the GPT models.

I have followed the steps from https://learn.microsoft.com/en-us/azure/api-management/azure-openai-enable-semantic-caching and re-checked multiple times to ensure that I haven't missed any step. Ideally, for the same request to the same GPT model sent again before the cache expiration time should return responses in much lesser time. This is done by policies configured on the API(https://learn.microsoft.com/en-us/azure/api-management/azure-openai-semantic-cache-lookup-policy and https://learn.microsoft.com/en-us/azure/api-management/azure-openai-semantic-cache-store-policy). Both my GPT3.5 Turbo and Embeddings model work when tested from Test section of APIM but from trace I see this error signifying that caching is not working.

Any help would be greatly appreciated!

Answer 1

I was able to fix the issue which was preventing caching. There were 2 issues preventing caching:

The cache was not set to 'Default'. I had previously added my Redis Cache from a region by selecting a region from the dropdown 'Use from' and then selecting the resource from the 'Cache instance' dropdown. The 'Use from' value has to be instead selected as 'Default' and the 'Cache instance' should be kept to 'Custom' value and the connection string has to be manually filled in (of the format "<endpoint-url-of-cache>,password=<access-key>,ssl=True,abortConnect=False" )

The custom url in the backend for embeddings API was missing 'embeddings-deployment/embeddings' in the custom url