Groundedness Detection Character Limit Documentation/API Discrepancy

Question

Groundedness Detection Character Limit Documentation/API Discrepancy

J C 5

We're currently assessing your Groundedness Detection API under reference (2024-09-15-preview).

I've found that any text input over 750 Characters results in a false result/doesn't work. The documentation lists the input limit to be 7,500 characters.

I'm observing the following behaviour:

When submitting a request with 750 Characters:
- A valid Groundedness Detection (true) is returned - Expected Behaviour.
When submitting a request with 751 Characters:
- A false Groundedness Detection (false) response is returned - Unexpected Behaviour.
When submitting a request with 7501 Characters:
- A text length error response is returned - Expected Behaviour.

Please would you be able to confirm if this is a bug in the API, or a Documentation issue.

Manas Mohanty 5,790 Microsoft External Staff Moderator

Hi J C

Could you share your sample code with sample inputs in private chat for repro'ing this issue.

As part of my repro, I added few lines which might trigger content filter for abuse detection to 700 words, but it is able to do the abuse detection.

Below was my code used in repro purpose.

import os
from azure.ai.contentsafety import ContentSafetyClient
from azure.core.credentials import AzureKeyCredential
from azure.core.exceptions import HttpResponseError
from azure.ai.contentsafety.models import AnalyzeTextOptions, TextCategory

def analyze_text():
    # analyze text
    key = "<contentsafetykey>"
    endpoint ="<endpointurl>"

    # Create an Azure AI Content Safety client
    client = ContentSafetyClient(endpoint, AzureKeyCredential(key))
    example_text = """The playground gives you options to tailor your chat experience. On the top menu, you can select Deploy to determine which model generates a response using the search results from your index. You choose the number of past messages to include as conversation history for future generated responses. Conversation history gives context to generate related responses but also consumes token usage. The input token progress indicator keeps track of the token count of the question you submit.

The Advanced settings on the left are runtime parameters, which give you control over retrieval and search relevant information from your data. A good use case is when you want to make sure responses are generated only based on your data or you find the model cannot generate a response based on existed information on your data.

Strictness determines the system's aggressiveness in filtering search documents based on their similarity scores. Setting strictness to 5 indicates that the system will aggressively filter out documents, applying a very high similarity threshold. Semantic search can be helpful in this scenario because the ranking models do a better job of inferring the intent of the query. Lower levels of strictness produce more verbose answers but might also include information that isn't in your index. This is set to 3 by default.

Retrieved documents is an integer that can be set to 3, 5, 10, or 20, and controls the number of document chunks provided to the large language model for formulating the final response. By default, this is set to 5.

When Limit responses to your data is enabled, the model attempts to only rely on your documents for responses. 

This is set to true by default. 

<added a spam content here>
    
    The importance of clear communication in all aspects of life cannot be overstated. Effective communication 
serves as the foundation for healthy relationships, professional success, and societal harmony. Whether it's verbal or written, 
the ability to express thoughts and ideas clearly can significantly impact outcomes. In workplaces, for instance, strong 
communication skills help teams collaborate effectively, reducing misunderstandings and ensuring projects progress smoothly. 
Leaders who communicate their vision with clarity and enthusiasm inspire their teams to work toward common goals. 

Beyond professional settings, good communication plays a pivotal role in personal relationships. Honest and open dialogue fosters 
trust and mutual respect among friends, family members, and partners. Miscommunication, however, can lead to unnecessary conflicts 
that strain relationships. Learning to listen actively and articulate feelings can prevent such misunderstandings, promoting harmony 
and stronger connections. For example, a parent who listens attentively to their child’s concerns can create an environment where 
the child feels valued and understood. Similarly, partners who share their perspectives openly are more likely to build a 
relationship based on trust and support.

In education, communication is an essential tool for both teaching and learning. Teachers convey complex concepts through effective 
communication, adapting their methods to suit different learning styles. Students, in turn, benefit from clear instructions and 
constructive feedback, enabling them to grasp subjects more effectively. Consider a scenario where a student struggles to understand 
a scientific principle. A teacher who uses analogies, diagrams, and simple language can bridge that gap, making learning accessible.

The digital age has transformed how we communicate, offering instant connectivity but also posing challenges. Social media platforms 
have become primary modes of interaction, allowing people to share ideas, news, and opinions globally. However, the brevity and 
informality often associated with online communication can sometimes lead to misinterpretations. While emojis and abbreviations 
help convey emotions, they lack the depth of face-to-face conversations. Moreover, the prevalence of misinformation online highlights 
the need for critical thinking and responsible sharing.

Clear communication is also vital in addressing societal challenges. Whether it's advocating for environmental conservation or raising 
awareness about public health issues, compelling communication can mobilize action. Activists use speeches, articles, and campaigns 
to educate the public and rally support. For instance, well-crafted messages about climate change have motivated individuals to 
adopt eco-friendly practices and demand policy changes. Similarly, during health crises like pandemics, transparent communication 
from authorities ensures the public stays informed and prepared.

One cannot ignore the cultural nuances in communication. Different societies have unique norms and practices that influence how 
messages are conveyed and received. What might be considered direct and honest in one culture could be perceived as blunt or rude 
in another. Understanding these cultural differences is essential for effective international relations and global cooperation. For 
example, in Japan, indirect communication and maintaining harmony are highly valued, whereas Western cultures often prioritize 
straightforwardness.

Communication skills can be honed through practice and reflection. Engaging in conversations with diverse individuals helps expand 
perspectives and improve adaptability. Public speaking workshops, writing courses, and listening exercises are excellent ways to 
develop proficiency. For example, attending a debate class can sharpen critical thinking and the ability to express arguments logically 
and persuasively. Similarly, journaling enhances written communication while providing an outlet for introspection.

Finally, technology offers innovative tools to improve communication. AI-powered translation apps, video conferencing software, and 
online collaborative platforms have revolutionized how people connect and collaborate. These tools break geographical barriers, 
allowing businesses to operate globally and individuals to maintain relationships across continents. For instance, a family spread 
across multiple countries can stay connected through regular video calls, fostering a sense of togetherness despite physical distance.

To conclude, communication is more than the exchange of words; it's about understanding and connection. Developing the ability to 
communicate effectively enriches every aspect of life, from personal relationships to societal progress. As technology advances and 
the world becomes more interconnected, mastering communication is increasingly crucial. It empowers individuals to share ideas, solve 
problems, and build meaningful connections in a rapidly changing world."""
    # Contruct request
    request = AnalyzeTextOptions(text=example_text)

    # Analyze text
    try:
        response = client.analyze_text(request)
    except HttpResponseError as e:
        print("Analyze text failed.")
        if e.error:
            print(f"Error code: {e.error.code}")
            print(f"Error message: {e.error.message}")
            raise
        print(e)
        raise

    hate_result = next(item for item in response.categories_analysis if item.category == TextCategory.HATE)
    self_harm_result = next(item for item in response.categories_analysis if item.category == TextCategory.SELF_HARM)
    sexual_result = next(item for item in response.categories_analysis if item.category == TextCategory.SEXUAL)
    violence_result = next(item for item in response.categories_analysis if item.category == TextCategory.VIOLENCE)

    if hate_result:
        print(f"Hate severity: {hate_result.severity}")
    if self_harm_result:
        print(f"SelfHarm severity: {self_harm_result.severity}")
    if sexual_result:
        print(f"Sexual severity: {sexual_result.severity}")
    if violence_result:
        print(f"Violence severity: {violence_result.severity}")

if __name__ == "__main__":
    analyze_text()

Thank you.

Manas Mohanty 5,790 Reputation points Microsoft External Staff Moderator

2025-03-26T11:19:16.03+00:00

Hi J C

We are still waiting to hear your inputs.

Thank you.
Manas Mohanty 5,790 Reputation points Microsoft External Staff Moderator

2025-03-27T05:03:28.2066667+00:00

Hi J C

We have not heard from you. Hope the pointers shared were useful to you.

Thank you.

J C 5

Hello @Manas Mohanty

See below code to reproduce the issue in question. Note the code you've provided isn't testing the product in question: Groundedness Detection.

Findings: Server side failure in processing text size requests between 750-7500 Characters.

Reference: Text input limit should be 7500 Chars. The API is failing at 750 Chars

import http.client
import json

endpoint = "[REDACTED].cognitiveservices.azure.com"
key = "[REDACTED]"

conn = http.client.HTTPSConnection(endpoint)

def callAPI(text):
    payload = json.dumps({
      "domain": "Generic",
      "task": "Summarization",
      "text": text,
      "groundingSources": ["Foxes are orange"],
      "reasoning": False
    })
    headers = {
      'Ocp-Apim-Subscription-Key': key,
      'Content-Type': 'application/json'
    }
    conn.request("POST", "/contentsafety/text:detectGroundedness?api-version=2024-09-15-preview", payload, headers)
    res = conn.getresponse()
    data = res.read()
    return(data.decode("utf-8"))

InputText = "Foxes are red"
Filler_749_chars = InputText + ("Z" * (749 - len(InputText)))
Filler_751_chars = InputText + ("Z" * (751 - len(InputText)))
Filler_7501_chars = InputText + ("Z" * (7501 - len(InputText)))



##Findings: Server side failure in processing text size requests between 750-7500 Characters.
##Reference: Text input limit should be 7500 Chars. The API is failing at 750 Chars https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview#input-requirements

## Reproduction

## 13 Chars
##Expect: Ungroundedness: True
##Recieved: Ungroundedness: True
##Result: Expected Behaviour 
print("13 Chars", callAPI(InputText))

print("##########################\n\n")

## 749 Chars
##Expect: Ungroundedness: True
##Recieved: Ungroundedness: True
##Result: Expected Behaviour 
print("749 Chars", callAPI(Filler_749_chars))

print("##########################\n\n")

## 751 Chars
##Expect: Ungroundedness: True
##Recieved: Ungroundedness: False
##Result: Unexpected Behaviour - Empty Response 
print("751 Chars", callAPI(Filler_751_chars))

print("##########################\n\n")

## 7501 Chars
##Expect: API Text Size Limit Error
##Recieved: API Text Size Limit Error
##Result: Expected Behaviour 
print("7501 Chars", callAPI(Filler_7501_chars))

Manas Mohanty 5,790 Reputation points Microsoft External Staff Moderator

2025-03-28T09:12:51.5866667+00:00

Hi J C

I am able to replicate the Discrepancy at 751 characters with above code and updating team internally for further actions.

Thank you
JAYA SHANKAR G S 3,960 Reputation points Microsoft External Staff Moderator

2025-04-01T06:42:59.81+00:00

Hello @J C ,

We are checking with internal team regarding the issue, will update you once we get the details.

Thank you

1 answer

Your answer

Manas Mohanty 5,790 Reputation points Microsoft External Staff Moderator

2025-03-26T11:19:16.03+00:00

Hi J C

We are still waiting to hear your inputs.

Thank you.
Manas Mohanty 5,790 Reputation points Microsoft External Staff Moderator

2025-03-27T05:03:28.2066667+00:00

Hi J C

We have not heard from you. Hope the pointers shared were useful to you.

Thank you.
Manas Mohanty 5,790 Reputation points Microsoft External Staff Moderator

2025-03-28T09:12:51.5866667+00:00

Hi J C

I am able to replicate the Discrepancy at 751 characters with above code and updating team internally for further actions.

Thank you
JAYA SHANKAR G S 3,960 Reputation points Microsoft External Staff Moderator

2025-04-01T06:42:59.81+00:00

Hello @J C ,

We are checking with internal team regarding the issue, will update you once we get the details.

Thank you

Answer 1

JAYA SHANKAR G S 3,960 Microsoft External Staff Moderator

Hello @J C ,

We have confirmed that this is a bug and are currently working with the product team to resolve the issue. We will provide an update once the fix has been implemented.
At this time, there is no estimated timeline for resolution.

Thank you for your patience and understanding.

Share via

Groundedness Detection Character Limit Documentation/API Discrepancy

1 answer

Your answer