Azure OpenAI API-- associate file with vector store on upload not working

Question

Hi. Using AzureOpenAI (client = AzureOpenAI())

I can query my vector store and I can upload files, but the files don't seem to actually associate with the vector store when I load the Azure AI Foundry/ check it in the assistant vector stores section.

I don't get an error, it's just that the files don't seem to be attached. Any idea if this is a bug?

This works:

`

Retrieve files from the vector store.

def get_vector_store_files(vector_store_id, limit=100): file_data = [] after = None

while True:
    try:
        logging.info(f"Attempting to fetch vector store files for {vector_store_id} with after={after}")
        response = client.vector_stores.files.list(vector_store_id, limit=limit, after=after)
    except Exception as e:
        logging.error(f"Error fetching vector store files: {e}")
        break

    for file in response.data:
        try:
            file_detail = client.files.retrieve(file.id)
            upload_time = datetime.fromtimestamp(file_detail.created_at)
            vs_filename = file_detail.filename
            file_key = vs_filename[-16:]  # use last 16 characters for matching
            file_data.append({
                "VectorStoreFileName": vs_filename,
                "FileKey": file_key,
                "VectorStoreUpload": upload_time,
                "VectorStoreFileID": file_detail.id
            })
            logging.info(f"Retrieved vector store file: {vs_filename}")
        except Exception as e:
            logging.error(f"Error retrieving details for file ID {file.id}: {e}")

    if getattr(response, 'has_more', False):
        after = response.data[-1].id
    else:
        break

return file_data`

this works too, but not putting the files on the VS itself.

`

def upload_file_to_vector_store(vector_store_id, file_path): try: with open(file_path, "rb") as f: # Use the file batch helper to upload and attach the file. file_batch = client.vector_stores.file_batches.upload_and_poll( vector_store_id=vector_store_id, files=[f] ) logging.info(f"File batch upload status: {file_batch.status}") logging.info(f"File batch counts: {file_batch.file_counts}") return file_batch except Exception as e: logging.error(f"Error uploading file '{file_path}' to vector store: {e}")

def fix_any_upload_issues(vector_store_id): """ Checks the vector store for files with a status of 'failed' and attempts to reattach them. Retries up to 5 times per file. """ try: files_page = client.vector_stores.files.list(vector_store_id) files = files_page.data except Exception as e: logging.error(f"Error listing files from vector store {vector_store_id}: {e}") return

# Filter files that have a 'failed' status.
failed_files = [f for f in files if getattr(f, "status", None) == "failed"]
logging.info(f"Initial failed files: {[f.id for f in failed_files]}")

for failed_file in failed_files:
    attempt = 0
    success = False

    while attempt < 5 and not success:
        attempt += 1
        logging.info(f"Attempt {attempt} for file {failed_file.id}")
        try:
            # Attempt to reattach the failed file to the vector store.
            # Note: Using the create() method here. Depending on your SDK version,
            # you might need to call client.vector_stores.files.create(...)
            client.vector_stores.files.create(
                vector_store_id, file_id=failed_file.id
            )
            # After the attempt, re-read the file list and check if this file still fails.
            updated_files_page = client.vector_stores.files.list(vector_store_id)
            updated_files = updated_files_page.data
            updated_failed_files = [
                f for f in updated_files if getattr(f, "status", None) == "failed"
            ]
            if not any(f.id == failed_file.id for f in updated_failed_files):
                success = True
                logging.info(f"Successfully reattached file {failed_file.id}")
            else:
                logging.info(f"File {failed_file.id} still in failed status after attempt {attempt}")
        except Exception as error:
            logging.error(
                f"Failed to reattach file {failed_file.id} on attempt {attempt}: {error}"
            )

logging.info("Finished processing failed files.")`

Answer 1

Hi Ollie Gooding

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer.

Ask: Azure OpenAI API-- associate file with vector store on upload not working

Solution: The issue is resolved. By the way that you finally figured out the real issue. You think it's a bug in the playground UI.

It's that the Vector Store in the Assistants playground just wasn't displaying all the files-- and still isn't. I had 200 files in there, but I click on Vector stores (preview) and reset view and there's only about 10.

The files were uploading all along (after figuring out the issue with .beta and not .beta

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

 

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

Thank You.

Answer 2

Here's the key takeaway from the latest logs:

1. Client structure check passed: 'client.vector_stores.files' found. - Good! This confirms that using client.vector_stores.files.list (and likely delete) is the correct path for your setup.
2. WARNING - Client object is missing 'beta.vector_stores.file_batches' used for upload_and_poll. - Problem Identified! This confirms our earlier finding that client.beta.vector_stores doesn't exist, and therefore the path needed for the upload_and_poll function as documented is missing.
3. ERROR - fmgem:249 - Client.beta object missing 'vector_stores' attribute. - This error happened inside the upload_files_batch_to_vector_store function when it tried to check for client.beta.vector_stores before making the call, confirming the warning.

Conclusion & Hypothesis:

It appears that for your specific environment (AzureOpenAI client, library v1.72.0, API version 2024-05-01-preview), the vector store operations are not under the beta attribute as shown in some documentation examples. Instead, they seem to be directly under client.vector_stores.

Since client.vector_stores.files exists (used for list/delete), let's hypothesize that the batch upload function might also be under this structure, perhaps as client.vector_stores.file_batches.upload_and_poll.

Next Step:

I will modify the upload_files_batch_to_vector_store function to use client.vector_stores.file_batches.upload_and_poll(...) instead of client.beta.vector_stores.file_batches.upload_and_poll(...). This makes the access pattern consistent for all vector store operations based on your logs.

Summary of Changes (v3.2 -> v3.3):

1. Upload Function Modified:
   • upload_files_batch_to_vector_store now checks for and calls client.vector_stores.file_batches.upload_and_poll(...).
   1. Client Checks Updated: The checks after client initialization now verify client.vector_stores.files and client.vector_stores.file_batches. It will log warnings if either is missing but only exits if both are missing.

Please run this version (azure_vector_store_sync_v3.3). This is our best attempt based on the logs showing client.vector_stores.files exists while client.beta.vector_stores... does not. Let's see if the upload call now succeeds using this adjusted path.Here's the key takeaway from the latest logs:

1. Client structure check passed: 'client.vector_stores.files' found. - Good! This confirms that using client.vector_stores.files.list (and likely delete) is the correct path for your setup.
2. WARNING - Client object is missing 'beta.vector_stores.file_batches' used for upload_and_poll. - Problem Identified! This confirms our earlier finding that client.beta.vector_stores doesn't exist, and therefore the path needed for the upload_and_poll function as documented is missing.
3. ERROR - fmgem:249 - Client.beta object missing 'vector_stores' attribute. - This error happened inside the upload_files_batch_to_vector_store function when it tried to check for client.beta.vector_stores before making the call, confirming the warning.

Conclusion & Hypothesis:

It appears that for your specific environment (AzureOpenAI client, library v1.72.0, API version 2024-05-01-preview), the vector store operations are not under the beta attribute as shown in some documentation examples. Instead, they seem to be directly under client.vector_stores.

Since client.vector_stores.files exists (used for list/delete), let's hypothesize that the batch upload function might also be under this structure, perhaps as client.vector_stores.file_batches.upload_and_poll.

Next Step:

I will modify the upload_files_batch_to_vector_store function to use client.vector_stores.file_batches.upload_and_poll(...) instead of client.beta.vector_stores.file_batches.upload_and_poll(...). This makes the access pattern consistent for all vector store operations based on your logs.

Summary of Changes (v3.2 -> v3.3):

1. Upload Function Modified:
   • upload_files_batch_to_vector_store now checks for and calls client.vector_stores.file_batches.upload_and_poll(...).
   1. Client Checks Updated: The checks after client initialization now verify client.vector_stores.files and client.vector_stores.file_batches. It will log warnings if either is missing but only exits if both are missing.

Please run this version (azure_vector_store_sync_v3.3). This is our best attempt based on the logs showing client.vector_stores.files exists while client.beta.vector_stores... does not. Let's see if the upload call now succeeds using this adjusted path.