Azure Prompt Flow with RankZephyr: How to handle Python 3.9 and 3.10 dependency conflicts?

Question

I am working with Azure Prompt Flow and attempting to integrate RankZephyr for document reranking. However, I am facing a Python version compatibility issue between Prompt Flow (Python 3.9) and RankZephyr (which requires Python 3.10+).

I am working on Azure Prompt Flow (in Azure AI Foundry) and started with the Multi-Round Q&A on Your Data prompt flow

What I Did

Changed the Index Lookup:

I replaced the built-in index lookup with LangChain’s AzureSearch() class:

        docs = vectorestore.similarity_search(

        query=query,

        k=TOP_K,

        search_type="hybrid",

        filters =  filter_query,

    )

I also added the required packages in requirements.txt.

Integrated RankZephyr for Reranking:

I want to pass the retrieved documents to RankZephyr for reranking.

Here’s the function I implemented:

from langchain.retrievers.contextual_compression import ContextualCompressionRetriever

from langchain_community.document_compressors.rankllm_rerank import RankLLMRerank

from langchain.schema import Document

from promptflow import tool

from langchain.schema import Document

from typing import List, Dict

TOP_n = 5

@tool

def rerankZephyr(input1: str, input2: List[Document]) -> List[Document]:

    compressor = RankLLMRerank(top_n=TOP_n, model="zephyr")

    compression_retriever = ContextualCompressionRetriever(

    base_compressor=compressor, base_retriever=input2

    )

    # Test con una query

    query = input1

    retrieved_docs = compression_retriever.get_relevant_documents(query)

    # Stampa i risultati con gli score di ranking

    #for doc in retrieved_docs:

    #    print(f"Score: {doc.metadata.get('score', 'N/A')} | Document: {doc.page_content}")

 return retrieve_docs

The Problem:

Azure Prompt Flow runs on Python 3.9, but RankZephyr requires Python 3.10+ (Link to the documentation).

This version mismatch causes dependency conflicts, preventing me from using RankZephyr inside Prompt Flow.

Where I find the above critical dependencies.

1. On Azure AI Foundry; when I click the 'Start Session' button, I receive the following error message: ERROR: Ignored the following versions that require a different Python version: 0.0.5 Requires-Python >=3.10; 0.0.6 Requires-Python >=3.10; 0.0.7 Requires-Python >=3.10; 0.1.0 Requires-Python >=3.10; 0.1.0 Requires-Python >=3.10
2. when using the VS Code Prompt Flow extension, I see the following prerequisite requirement: Python 3.9 is required.

Question:

What is the best way to integrate RankZephyr (Python 3.10) with Azure Prompt Flow (Python 3.9)?

1. Upgrading Python in Azure Prompt Flow adding Python>=3.10 in requirements.txt;
2. Using a separate Python 3.10 environment, but I am not sure how to integrate it with Prompt Flow.
3. Exploring Docker containers; thinking of running RankZephyr in a separate container, but unsure how to pass the data between Prompt Flow and the reranker efficiently.

Answer 1

Hi Matteo Doni,

You're facing a Python version mismatch issue between Azure Prompt Flow (Python 3.9) and RankZephyr (Python 3.10+). Since upgrading Prompt Flow's Python version directly isn't an option, here are the best approaches to resolve this:

Solution 1: Use an External Python 3.10+ Service (Recommended):

The best way to integrate RankZephyr is to run it in a separate Python 3.10+ environment (e.g., an Azure Function, Azure Container App, or a simple REST API) and call it from Prompt Flow via an API request.

Steps:

Create a Python 3.10+ environment on an Azure Function or Azure Container App.

Expose RankZephyr as a REST API (FastAPI/Flask).

Call this API from Prompt Flow after retrieving documents.

Implementation:

1.Create a FastAPI Server for RankZephyr (Python 3.10+)

Deploy this on Azure Functions or an Azure VM.

from fastapi import FastAPI
from typing import List
from langchain.retrievers.contextual_compression import ContextualCompressionRetriever
from langchain_community.document_compressors.rankllm_rerank import RankLLMRerank
from langchain.schema import Document

app = FastAPI()

TOP_N = 5
reranker = RankLLMRerank(top_n=TOP_N, model="zephyr")

@app.post("/rerank")
async def rerank_documents(query: str, documents: List[dict]):
    # Convert JSON to LangChain Document format
    docs = [Document(page_content=d["page_content"], metadata=d.get("metadata", {})) for d in documents]

    # Apply reranking
    compression_retriever = ContextualCompressionRetriever(base_compressor=reranker, base_retriever=docs)
    reranked_docs = compression_retriever.get_relevant_documents(query)

    return [{"content": doc.page_content, "score": doc.metadata.get("score", "N/A")} for doc in reranked_docs]

Deploy this API (e.g., on an Azure Function or a VM with Python 3.10).

Test with cURL or Postman:

curl -X POST "http://your-api-endpoint/rerank" \
     -H "Content-Type: application/json" \
     -d '{"query": "Your search query", "documents": [{"page_content": "Doc 1"}, {"page_content": "Doc 2"}]}'

2.Call This API from Azure Prompt Flow (Python 3.9)

Modify your Prompt Flow function:

import requests
from typing import List, Dict
from langchain.schema import Document
from promptflow import tool

RERANKER_API_URL = "http://your-api-endpoint/rerank"

@tool
def rerankZephyr(input1: str, input2: List[Document]) -> List[Document]:
    payload = {
        "query": input1,
        "documents": [{"page_content": doc.page_content, "metadata": doc.metadata} for doc in input2]
    }
    response = requests.post(RERANKER_API_URL, json=payload)
    reranked_docs = response.json()
    return [Document(page_content=d["content"], metadata={"score": d["score"]}) for d in reranked_docs]

Solution 2: Use a Docker Container

If you can't deploy a separate API, you can run RankZephyr inside a Docker container and connect it to Azure Prompt Flow.

Steps:

1.Create a Dockerfile with Python 3.10+:

FROM python:3.10
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "rerank_service.py"]

2.Run Docker locally or deploy on Azure Container Apps.

3.Use the same API-based approach as above.

Hope this helps. Do let us know if you any further queries.  

------------- 

If this answers your query, do click Accept Answer and Yes for was this answer helpful.

Thank you.