Azure AI Search
An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.
937 questions
Data preprocessing tips and tricks before indexing
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 71%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDK%3C/text%3E%3C/svg%3E)
Dezső Kántor
0
Reputation points
I am in the process of developing an AI-based QA chatbot that will provide answers using my own data. My question is fairly broad. I am working with various data formats, including HTML files, PDFs, and Markdown files. What is the best way to preprocess this data before indexing and vectorizing to ensure optimal input for AI search indexing? I found a promising approach by using OpenAI to convert the files into JSON objects, which I then used as input for the index. What are the best practices for data preprocessing? Additionally, what is the most effective input format for AI search indexing?
Sign in to answer