How to best structure skillsets and indexing flow for chunking, embedding, and querying uploaded files via Azure AI Search and OpenAI?

Question

How to best structure skillsets and indexing flow for chunking, embedding, and querying uploaded files via Azure AI Search and OpenAI?

Mo Ibrahim 0

I'm building a pipeline to upload academic files (e.g., PDFs), split and embed them using skillsets, and then query them via Azure AI Search + OpenAI. My goal is to generate academic questions using document content only.

Current Flow

Upload a file to Blob Storage with metadata (e.g., userId, fileName).
Indexer runs with a skillset that includes:
- SplitSkill (mode: pages, max length 4000, overlap 300)
- AzureOpenAIEmbeddingSkill for generating vector embeddings.
The index projection outputs:
- chunk, chunk_vector, fileName, userId, blobPath
I then use Azure OpenAI with azure_search as the data_source:
```
   {
```
The final prompt sent to OpenAI (via /chat/completions) includes:
- A system prompt like:
  "You are a virtual assistant for academic instructors."
- Instructions to return {Questions:[]} if the document has insufficient or unrelated content.

What I Need Help With

Can I further reduce the size of each chunk before embedding by ignoring irrelevant characters or whitespace before the SplitSkill? I want to avoid wasting tokens on junk formatting.
Can I apply character normalization (e.g., regex, spacing, junk removal) in a single skill, ideally before splitting?
Is there a more efficient or alternative approach to achieve this overall process (splitting, embedding, indexing, and querying)? I’m open to other architectural patterns or using different Azure tools.

Mo Ibrahim 0

this is the shape of json send as datasource to open ai
{

"type": "azure_search",

"parameters": {

"endpoint": "<search-endpoint>",

"index_name": "<index-name>",

"authentication": {

  "type": "api_key",

  "key": "<search-key>"

},

"in_scope": true,

"filter": "userId eq '<userId>' and fileName eq '<fileName>'",

"fields_mapping": {

  "content_fields": ["chunk"],

  "vector_fields": ["chunk_vector"]

}

}

Bhargavi Naragani 6,535 Reputation points Microsoft External Staff Moderator

2025-07-03T07:11:09.8533333+00:00
Hi Mo Ibrahim,

Can I further reduce the size of each chunk before embedding by ignoring irrelevant characters or whitespace before the SplitSkill? I want to avoid wasting tokens on junk formatting.

Yes! To avoid wasting tokens and improve embedding quality, it's highly recommended to clean up the text before splitting. You can do this using the TextSplitSkill combined with custom regex or pre-processing skills.

Custom Text Cleanup via CustomSkill: Use an Azure Function or Logic App as a skill to clean: Excess whitespace, Redundant headers/footers, Special characters or junk formatting (e.g., \t, page numbers, section headers) Custom Skills Overview

Can I apply character normalization (e.g., regex, spacing, junk removal) in a single skill, ideally before splitting?

Yes, you can apply normalization logic in a single Custom Web API Skill. For example, build a lightweight Azure Function with logic like:

import re def clean_text(text): # remove extra spaces text = re.sub(r'\s+', ' ', text) # remove page numbers or special formatting text = re.sub(r'Page \d+ of \d+', '', text) return text.strip()

Configure this as a preprocessing skill before SplitSkill.

Ensure your skillset chain looks like: [Custom Text Cleanup Skill] ➜ [SplitSkill] ➜ [AzureOpenAIEmbeddingSkill]

Create a Custom Skill in Azure

Is there a more efficient or alternative approach to achieve this overall process (splitting, embedding, indexing, and querying)? I’m open to other architectural patterns or using different Azure tools.

Your current architecture is generally well-structured, but here are some key optimizations:

Use SplitSkill with mode "pages" only if each page is semantically meaningful. Otherwise, prefer mode: "textChunks" for smarter splitting based on sentence boundaries.

Store original + cleaned content, you can keep both raw_chunk and cleaned_chunk for transparency/debugging or future embedding improvements.

Indexing Improvements: Add embedding_timestamp to track re-embedding needs. Use Incremental Indexing to update only new or changed files.

Prompting Enhancements for OpenAI: You can use "in_scope": true + "semantic_configuration": "my-semantic-config" to boost relevance via semantic ranker.

Optionally use Azure Document Intelligence if you need structured data extraction before chunking, especially useful for academic papers with tables/figures. Azure AI Document Intelligence

Hope this helps, if you have any further concerns or queries, please feel free to reach out to us.

Mo Ibrahim 0

Thank you for your clear and detailed response.

However, I'm still facing some issues with integrating Azure Search with OpenAI. Specifically, when I search a topic directly using Azure Search, I receive lots of results. But when I use the same configuration through OpenAI, it almost always fails to retrieve any data.

Here is the configuration I'm using:

var dataSource = fileName == null ? null : new[]
   {
        new
        {
            type = "azure_search",
            parameters = new
            {
                endpoint  = Configurations.AISearchEndpoint,
                index_name = Configurations.AISearchIndexName,
                authentication = new
                {
                    type = "api_key",
                    key  = Configurations.AISearchKey
                },
                top_n_documents = 15,
                query_type = "vector_semantic_hybrid",
                semantic_configuration_name = Constants.AzureSearchConstants.SemanticConfigurationName,
                in_scope = true,
                filter = $"userId eq '{userId}' and fileName eq '{fileName}'",
                fields_mapping = new
                {
                    content_fields = new[] { "chunk" },
                    vector_fields = new[] { "chunk_vector" }
                }
            }
        }
    };

If you have any suggestions on what might be going wrong or what could be adjusted to make the retrieval work correctly with OpenAI, I’d really appreciate the guidance.

Manas Mohanty 6,285 Reputation points Microsoft External Staff Moderator

2025-07-08T05:19:40.67+00:00

Hi Mo Ibrahim

Hope you appreciated our inputs. Please let us know if you were able to get improved results with above optimization.

Thank you
Manas Mohanty 6,285 Reputation points Microsoft External Staff Moderator

2025-07-09T01:10:53.4566667+00:00

Hi Mo Ibrahim

We could not hear from you. Hope you found below pointers on Azure OpenAI useful

Thank you.

1 answer

Your answer

Mo Ibrahim 0 Reputation points

2025-07-02T11:56:20.8233333+00:00

this is the shape of json send as datasource to open ai
{

"type": "azure_search",

"parameters": {

"endpoint": "<search-endpoint>", "index_name": "<index-name>", "authentication": { "type": "api_key", "key": "<search-key>" }, "in_scope": true, "filter": "userId eq '<userId>' and fileName eq '<fileName>'", "fields_mapping": { "content_fields": ["chunk"], "vector_fields": ["chunk_vector"] }

}

}
Mo Ibrahim 0 Reputation points

2025-07-06T07:57:25.2433333+00:00

Thank you for your clear and detailed response.

However, I'm still facing some issues with integrating Azure Search with OpenAI. Specifically, when I search a topic directly using Azure Search, I receive lots of results. But when I use the same configuration through OpenAI, it almost always fails to retrieve any data.

Here is the configuration I'm using:

var dataSource = fileName == null ? null : new[] { new { type = "azure_search", parameters = new { endpoint = Configurations.AISearchEndpoint, index_name = Configurations.AISearchIndexName, authentication = new { type = "api_key", key = Configurations.AISearchKey }, top_n_documents = 15, query_type = "vector_semantic_hybrid", semantic_configuration_name = Constants.AzureSearchConstants.SemanticConfigurationName, in_scope = true, filter = $"userId eq '{userId}' and fileName eq '{fileName}'", fields_mapping = new { content_fields = new[] { "chunk" }, vector_fields = new[] { "chunk_vector" } } } } };

If you have any suggestions on what might be going wrong or what could be adjusted to make the retrieval work correctly with OpenAI, I’d really appreciate the guidance.
Manas Mohanty 6,285 Reputation points Microsoft External Staff Moderator

2025-07-08T05:19:40.67+00:00

Hi Mo Ibrahim

Hope you appreciated our inputs. Please let us know if you were able to get improved results with above optimization.

Thank you
Manas Mohanty 6,285 Reputation points Microsoft External Staff Moderator

2025-07-09T01:10:53.4566667+00:00

Hi Mo Ibrahim

We could not hear from you. Hope you found below pointers on Azure OpenAI useful

Thank you.

Answer 1

Hi Mo Ibrahim

It seems you are able to pull data correctly from AI search queries, but it is not pulling the data result correctly from Azure OpenAI side.

Could you do the following and let us know.

Allows Azure OpenAI URLs, storage URLs from AI search CORS policy (Cross origin resource sharing).
Add Azure OpenAI and trusted resource from AI search side - https://learn.microsoft.com/en-us/azure/search/service-configure-firewall
lower temperature ( to 0.3 or .2) and increase Top_N and Top_k
Improve system instruction with specific needs on desired outputs (more examples and failovers)
Check official dotnet SDK code for add your own data scenario.

https://learn.microsoft.com/en-us/azure/ai-foundry/openai/concepts/use-your-data?tabs=ai-search%2Ccopilot

On AI search side, you can opt or hybrid approach (Semantic+Vector indexing)

Please share your configuration details on Azure AI search (CORS policy) and Azure OpenAI (Top_N, Top_K, Temperature, Chunk Size) side for further assistance.

Thank you.

Share via

How to best structure skillsets and indexing flow for chunking, embedding, and querying uploaded files via Azure AI Search and OpenAI?

1 answer

Your answer