Support for PDF files as base64 of the OpenAI SDK

Question

Support for PDF files as base64 of the OpenAI SDK

Christian Korherr 0

Will there be support for PDF files as base64 for the OpenAI SDK?

export namespace ChatCompletionContentPart {
  /**
   * Learn about [file inputs](https://platform.openai.com/docs/guides/text) for text
   * generation.
   */
  export interface File {
    file: File.File;

    /**
     * The type of the content part. Always `file`.
     */
    type: 'file';
  }

  export namespace File {
    export interface File {
      /**
       * The base64 encoded file data, used when passing the file to the model as a
       * string.
       */
      file_data?: string;

      /**
       * The ID of an uploaded file to use as input.
       */
      file_id?: string;

      /**
       * The name of the file, used when passing the file to the model as a string.
       */
      file_name?: string;
    }
  }
}

If I use it I'm getting this error:

{
  "error": {
    "message": "Unknown parameter: 'messages[2].content[0].file.file_name'.",
    "type": "invalid_request_error",
    "param": "messages[2].content[0].file.file_name",
    "code": "unknown_parameter"
  }
}

Manas Mohanty 3,020 Microsoft External Staff

Hi Christian Korherr

Ideally, we use base64 encoding for images and can use pdfs directly.

Encoding pdfs to base64 format will need another decoding to make the pdf contents readable as pdfs need to decrypted and decoded prior as part of requirements.

Currently it is not supported for OpenAI Apis and suggesting decoding in output message with below python code.

The provided text is a base64-encoded PDF file. Decoding it reveals a PDF document that is likely to contain various types of content such as text, images, or graphics. \n\nTo analyze the content effectively, the following steps can be performed:\n\n1. Decode the Base64: The first step is to decode the base64-encoded string to retrieve the actual binary data of the PDF file.\n\n2. Extract Text: Utilize PDF parsing libraries (such as PyPDF2, pdfminer, or PyMuPDF in Python) to extract text from the decoded PDF file.\n\n3. Analyze the Extracted Text: Once the text is extracted, various analyses can be performed, such as:\n

import os
import base64
from openai import AzureOpenAI  

# Environment variables (Replace these values if needed)
endpoint = os.getenv("ENDPOINT_URL", "<endpointname>")  
deployment = os.getenv("DEPLOYMENT_NAME", "gpt-4o-mini")  
subscription_key = os.getenv("AZURE_OPENAI_API_KEY", "<apikey>")  

# Initialize Azure OpenAI Service client with key-based authentication    
client = AzureOpenAI(
    azure_endpoint=endpoint,
    api_key=subscription_key,
    api_version="2024-05-01-preview",
)

# Base64 encode the PDF file
PDF_PATH = "./testdoc.pdf"  # Specify the path to your PDF file
try:
    with open(PDF_PATH, "rb") as pdf_file:
        # Encode the file as base64
        base64_encoded_pdf = base64.b64encode(pdf_file.read()).decode("utf-8")
except FileNotFoundError:
    print("Error: PDF file not found. Ensure the correct path is provided.")
    exit()

# Prepare the chat prompt
chat_prompt = [
    {
        "role": "system",
        "content": "You are analyzing content from base64-encoded files. Extract the text from the encoded file and perform analysis."
    },
    {
        "role": "user",
        "content": f"Here is a base64-encoded PDF file: {base64_encoded_pdf}"
    }
]

# Include speech result if speech is enabled  
messages = chat_prompt  

# Generate the completion  
try:
    completion = client.chat.completions.create(
        model=deployment,
        messages=messages,
        max_tokens=800,
        temperature=0.7,
        top_p=0.95,
        frequency_penalty=0,
        presence_penalty=0,
        stop=None,
        stream=False
    )
    print(completion.to_json())
except Exception as e:
    print(f"Error during API call: {e}")

Hope you understand the product limitation.

Thank you.

Christian Korherr 0 Reputation points

2025-03-26T14:18:10.7333333+00:00

What I'm looking for is exactly this functionality that exists only for the OpenAI API, but is not supported by Azure OpenAI: https://platform.openai.com/docs/guides/pdf-files#base64-encoded-files

How it works To help models understand PDF content, we put into the model's context both the extracted text and an image of each page. The model can then use both the text and the images to generate a response. This is useful, for example, if diagrams contain key information that isn't in the text.

While the commonly used OpenAI SDK provides the input file type it can only be used with OpenAI directly but not with Azure OpenAI.
Manas Mohanty 3,020 Reputation points Microsoft External Staff

2025-03-26T16:01:25.3633333+00:00

Ok Christian Korherr

Thank you for clearing it out.

Could you post this idea/feature on feedback forum for OpenAI to be considered by product team.Thank you.

1 answer

Your answer

Christian Korherr 0 Reputation points

2025-03-26T14:18:10.7333333+00:00

What I'm looking for is exactly this functionality that exists only for the OpenAI API, but is not supported by Azure OpenAI: https://platform.openai.com/docs/guides/pdf-files#base64-encoded-files

How it works To help models understand PDF content, we put into the model's context both the extracted text and an image of each page. The model can then use both the text and the images to generate a response. This is useful, for example, if diagrams contain key information that isn't in the text.

While the commonly used OpenAI SDK provides the input file type it can only be used with OpenAI directly but not with Azure OpenAI.
Manas Mohanty 3,020 Reputation points Microsoft External Staff

2025-03-26T16:01:25.3633333+00:00

Ok Christian Korherr

Thank you for clearing it out.

Could you post this idea/feature on feedback forum for OpenAI to be considered by product team.Thank you.

Answer 1

Christian Korherr 0

For everyone interested, you can vote for the feature here: https://feedback.azure.com/d365community/idea/5c414e8f-d40a-f011-a4de-6045bdbc7eba

Share via

Support for PDF files as base64 of the OpenAI SDK

1 answer

Your answer