Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,645 questions
How to show citations?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 68%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAF%3C/text%3E%3C/svg%3E)
Aftab Farooqi, PhD
0
Reputation points
We are using Azure Cognitive Services, and our goal is to use RAG with PDF documents to answer technical questions. We are facing difficulties with getting citations to work as intended. When using the Azure extensions with cognitive services, we get back 5 "citations". These citations are not single sentences or even relevant paragraphs, but the entire chunk in the database. we wish to highlight only the relevant information in the document, which we have in a side pane next to the chat.
our current solution is to provide the User's chat message, the answer, and hand back the previously attached citation chunk and then request the LLM extract the most relevant sentence. This works around 70% of the time, but it seems like the LLM is unable to find the same information, from the same referenced chunk a second time around 30%.
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 74%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAM%3C/text%3E%3C/svg%3E)
AshokPeddakotla-MSFT 30,511 Reputation points2024-05-29T08:25:22.6033333+00:00 Aftab Farooqi, PhD Greetings & Welcome to Microsoft Q&A forum!
One possible reason for this could be that the LLM is not able to consistently identify the most relevant sentence from the citation chunk due to the variability in the structure and content of the PDF documents. Another reason could be that the LLM is not being trained on a large enough dataset to accurately identify the most relevant sentence.
Did you could consider training it on a larger dataset of PDF documents that are similar to the ones you are working with?
-
AshokPeddakotla-MSFT 30,511 Reputation points
2024-06-11T01:52:46.51+00:00 Aftab Farooqi, PhD Did you get a chance to see my response? Do let me know if you need any further help.
Sign in to comment