This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 72%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
Hello,
We used Azure GPT-4 Turbo-2024-04-09, and the test results differed from those of OpenAI. The responses from OpenAI were longer and clearer than those from Azure. This issue occurred when the GPT-4 Turbo preview was released.
Thank you.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 74%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAM%3C/text%3E%3C/svg%3E)
d0ng Greetings!
We used Azure GPT-4 Turbo-2024-04-09, and the test results differed from those of OpenAI. The responses from OpenAI were longer and clearer than those from Azure. This issue occurred when the GPT-4 Turbo preview was released.
Could you share any specific scenario where you have observed the differences?
Yes, I can send my test prompt and result to you by email, thanks.
Sure, please see private message.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 86%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMR%3C/text%3E%3C/svg%3E)
Seeing similar issue, see the min reply column (minimum number of completion tokens returned out of the three tries). With prompt: "explain ping", max_tokens: 300 returns from OpenAI direct always use all the 300 tokens, while from Azure they are always <200 tokens.
Reply from Azure:
{
"id": "chatcmpl-9KhixfZoTE4cUEg12yoSybg5viEyu",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"message": {
"content": "Ping is a network diagnostic tool used to test the connectivity between two devices in a network. It operates by sending an In
ternet Control Message Protocol (ICMP) echo request to a specified IP address and waits for an echo reply to measure the round-trip time it takes
for the request to reach the target and for the response to return. This measurement helps determine the network's latency and packet loss, which
can identify issues related to network speed or reliability. Ping is commonly used to troubleshoot and verify network connections and can be execu
ted from the command line or terminal on most operating systems.",
"role": "assistant",
"function_call": null,
"tool_calls": null
}
}
],
"created": 1714721663,
"model": "gpt-4-turbo-2024-04-09",
"object": "chat.completion",
"system_fingerprint": "fp_f5ce1e47b6",
"usage": {
"completion_tokens": 116,
"prompt_tokens": 9,
"total_tokens": 125
}
}
Reply from OpenAI direct:
{
"id": "chatcmpl-9Khixs525kvE7rSso5SomvHu3Glvn",
"choices": [
{
"finish_reason": "length",
"index": 0,
"logprobs": null,
"message": {
"content": "The term \"ping\" is used in the context of computer networks to describe a diagnostic tool that verifies the connectivity sta
tus between the sending computer and a specified destination computer or network device. The name \"ping\" comes from the sound of the sonar echo
as it bounces off an object in submarine communications. \n\nHere\u2019s a step-by-step breakdown of how ping works:\n\n1. **Ping Command:** A use
r or a system administrator enters the ping command followed by the hostname or the IP address of the destination device into a command-line inter
face (CLI) such as the Terminal in macOS and Linux, or Command Prompt in Windows.\n\n2. **ICMP ECHO Request:** The tool sends an ICMP (Internet Co
ntrol Message Protocol) Echo Request message to the network address of the specified host. This message asks the receiving device to send back a r
eply.\n\n3. **ICMP ECHO Reply:** If the destination device is available and configured to respond to ping requests, it will send back an ICMP Echo
Reply.\n\n4. **Receipt and Analysis:** The originating host receives the ICMP Echo Replies and calculates the time it took for the round trip. Th
is delay is usually displayed in milliseconds and is termed as latency. \n\n5. **Output Information:** The output of a ping command provides sever
al valuable pieces of information including:\n - The IP address of the target host.\n - The size of the ICMP messages sent and received.\n -
The time each ping took (Round-Trip Time - RTT).\n ",
"role": "assistant",
"function_call": null,
"tool_calls": null
}
}
],
"created": 1714721663,
"model": "gpt-4-turbo-2024-04-09",
"object": "chat.completion",
"system_fingerprint": "fp_ea6eb70039",
"usage": {
"completion_tokens": 300,
"prompt_tokens": 9,
"total_tokens": 309
}
}
This is fully reproducible with various prompts and when issuing then multiple times.
Another prompt which consistently results in far fewer tokens returned by Azure vs. OpenAI direct- prompt: 'hello world in c++', max_tokens: 300
Azure:
{
"id": "chatcmpl-9KhpyL2hQJE7ymVWcl0urTz1Vz0oF",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"message": {
"content": "#include <iostream>\nusing namespace std;\n\nint main() {\n cout << \"Hello world!\" << endl;\n return 0;\n}",
"role": "assistant",
"function_call": null,
"tool_calls": null
}
}
],
"created": 1714722098,
"model": "gpt-4-turbo-2024-04-09",
"object": "chat.completion",
"system_fingerprint": "fp_f5ce1e47b6",
"usage": {
"completion_tokens": 28,
"prompt_tokens": 12,
"total_tokens": 40
}
}
OpenAI:
{
"id": "chatcmpl-9Khpyo7zpXGwLVPwTW8lshyPldbQp",
"choices": [
{
"finish_reason": "length",
"index": 0,
"logprobs": null,
"message": {
"content": "Hello! To write a simple \"Hello, World!\" program in C++, you can use the following code. This program will display the messa
ge \"Hello, World!\" in the console when run.\n\nHere\u2019s how the code looks:\n\n```cpp\n#include <iostream>\n\nint main() {\n // Print \"He
llo, World!\" to the console\n std::cout << \"Hello, World!\" << std::endl;\n return 0;\n}\n```\n\n### Breakdown of the Code:\n\n1. **`#incl
ude <iostream>`**:\n - This line includes the Standard Input Output stream library which contains definitions for input/output operations. `std:
:cout` used in the program is part of this library.\n\n2. **`int main()`**:\n - This is the main function where the execution of the program beg
ins. It must return an integer, where `0` typically signifies successful completion.\n\n3. **`std::cout << \"Hello, World!\" << std::endl;`**:\n
- `std::cout` is used to output (\"print\") data to the standard output device, usually the console.\n - `\"Hello, World!\"` is the string that
is being output.\n - `<<` is the insertion operator used to pass the string to `std::cout`.\n - `std::endl` inserts a newline character and f
lushes the stream, ensuring that all output is properly displayed.\n\n4. **`return 0;`**",
"role": "assistant",
"function_call": null,
"tool_calls": null
}
}
],
"created": 1714722098,
"model": "gpt-4-turbo-2024-04-09",
"object": "chat.completion",
"system_fingerprint": "fp_ea6eb70039",
"usage": {
"completion_tokens": 300,
"prompt_tokens": 12,
"total_tokens": 312
}
}
With more complex queries (with system message, conversatioon history, etc.) the new model from Azure tends to forget to actually include a code (asking coding questions, and it replies like 'here is an updated code.' but it doesn't actually include the code in its reply and I have to ask it again. OpenAI had a similar issue back in Nov which they have fixed. Didn't have this issue (only sporadically) with the recent preview models from Azure nor OpenAI, nor the OpenAI/gpt-turbo-2024-04-09 behaves this way.
Azure model in Sweden-Central. Same issue in East-US-2 though.
Seeing similar issue, see the min reply column (minimum number of completion tokens returned out of the three tries). With prompt: "explain ping", max_tokens: 300 returns from OpenAI direct always use all the 300 tokens, while from Azure they are always <200 tokens.
Reply from Azure:
{
"id": "chatcmpl-9KhixfZoTE4cUEg12yoSybg5viEyu",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"message": {
"content": "Ping is a network diagnostic tool used to test the connectivity between two devices in a network. It operates by sending an In
ternet Control Message Protocol (ICMP) echo request to a specified IP address and waits for an echo reply to measure the round-trip time it takes
for the request to reach the target and for the response to return. This measurement helps determine the network's latency and packet loss, which
can identify issues related to network speed or reliability. Ping is commonly used to troubleshoot and verify network connections and can be execu
ted from the command line or terminal on most operating systems.",
"role": "assistant",
"function_call": null,
"tool_calls": null
}
}
],
"created": 1714721663,
"model": "gpt-4-turbo-2024-04-09",
"object": "chat.completion",
"system_fingerprint": "fp_f5ce1e47b6",
"usage": {
"completion_tokens": 116,
"prompt_tokens": 9,
"total_tokens": 125
}
}
Reply from OpenAI direct:
{
"id": "chatcmpl-9Khixs525kvE7rSso5SomvHu3Glvn",
"choices": [
{
"finish_reason": "length",
"index": 0,
"logprobs": null,
"message": {
"content": "The term \"ping\" is used in the context of computer networks to describe a diagnostic tool that verifies the connectivity sta
tus between the sending computer and a specified destination computer or network device. The name \"ping\" comes from the sound of the sonar echo
as it bounces off an object in submarine communications. \n\nHere\u2019s a step-by-step breakdown of how ping works:\n\n1. **Ping Command:** A use
r or a system administrator enters the ping command followed by the hostname or the IP address of the destination device into a command-line inter
face (CLI) such as the Terminal in macOS and Linux, or Command Prompt in Windows.\n\n2. **ICMP ECHO Request:** The tool sends an ICMP (Internet Co
ntrol Message Protocol) Echo Request message to the network address of the specified host. This message asks the receiving device to send back a r
eply.\n\n3. **ICMP ECHO Reply:** If the destination device is available and configured to respond to ping requests, it will send back an ICMP Echo
Reply.\n\n4. **Receipt and Analysis:** The originating host receives the ICMP Echo Replies and calculates the time it took for the round trip. Th
is delay is usually displayed in milliseconds and is termed as latency. \n\n5. **Output Information:** The output of a ping command provides sever
al valuable pieces of information including:\n - The IP address of the target host.\n - The size of the ICMP messages sent and received.\n -
The time each ping took (Round-Trip Time - RTT).\n ",
"role": "assistant",
"function_call": null,
"tool_calls": null
}
}
],
"created": 1714721663,
"model": "gpt-4-turbo-2024-04-09",
"object": "chat.completion",
"system_fingerprint": "fp_ea6eb70039",
"usage": {
"completion_tokens": 300,
"prompt_tokens": 9,
"total_tokens": 309
}
}
This is fully reproducible with various prompts and when issuing then multiple times.
Another prompt which consistently results in far fewer tokens returned by Azure vs. OpenAI direct- prompt: 'hello world in c++', max_tokens: 300
Azure:
{
"id": "chatcmpl-9KhpyL2hQJE7ymVWcl0urTz1Vz0oF",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"message": {
"content": "#include <iostream>\nusing namespace std;\n\nint main() {\n cout << \"Hello world!\" << endl;\n return 0;\n}",
"role": "assistant",
"function_call": null,
"tool_calls": null
}
}
],
"created": 1714722098,
"model": "gpt-4-turbo-2024-04-09",
"object": "chat.completion",
"system_fingerprint": "fp_f5ce1e47b6",
"usage": {
"completion_tokens": 28,
"prompt_tokens": 12,
"total_tokens": 40
}
}
OpenAI:
{
"id": "chatcmpl-9Khpyo7zpXGwLVPwTW8lshyPldbQp",
"choices": [
{
"finish_reason": "length",
"index": 0,
"logprobs": null,
"message": {
"content": "Hello! To write a simple \"Hello, World!\" program in C++, you can use the following code. This program will display the messa
ge \"Hello, World!\" in the console when run.\n\nHere\u2019s how the code looks:\n\n```cpp\n#include <iostream>\n\nint main() {\n // Print \"He
llo, World!\" to the console\n std::cout << \"Hello, World!\" << std::endl;\n return 0;\n}\n```\n\n### Breakdown of the Code:\n\n1. **`#incl
ude <iostream>`**:\n - This line includes the Standard Input Output stream library which contains definitions for input/output operations. `std:
:cout` used in the program is part of this library.\n\n2. **`int main()`**:\n - This is the main function where the execution of the program beg
ins. It must return an integer, where `0` typically signifies successful completion.\n\n3. **`std::cout << \"Hello, World!\" << std::endl;`**:\n
- `std::cout` is used to output (\"print\") data to the standard output device, usually the console.\n - `\"Hello, World!\"` is the string that
is being output.\n - `<<` is the insertion operator used to pass the string to `std::cout`.\n - `std::endl` inserts a newline character and f
lushes the stream, ensuring that all output is properly displayed.\n\n4. **`return 0;`**",
"role": "assistant",
"function_call": null,
"tool_calls": null
}
}
],
"created": 1714722098,
"model": "gpt-4-turbo-2024-04-09",
"object": "chat.completion",
"system_fingerprint": "fp_ea6eb70039",
"usage": {
"completion_tokens": 300,
"prompt_tokens": 12,
"total_tokens": 312
}
}
With more complex queries (with system message, conversatioon history, etc.) the new model from Azure tends to forget to actually include a code (asking coding questions, and it replies like 'here is an updated code.' but it doesn't actually include the code in its reply and I have to ask it again. OpenAI had a similar issue back in Nov which they have fixed. Didn't have this issue (only sporadically) with the recent preview models from Azure nor OpenAI, nor the OpenAI/gpt-turbo-2024-04-09 behaves this way.
Azure model in Sweden-Central. Same issue in East-US-2 though.
I conducted some tests with the GPT-4 turbo-2024-04-09 on both OpenAI and Azure APIs. I found that the Azure is less responsive than OpenAI.
This issue may be caused by differences in the ‘system prompt’ between OpenAI and Azure. What works well with OpenAI may not work with Azure.
The solution is to modify the prompt to include clearer instructions, requesting the model to generate a more detailed and accurate response.
Thanks for sharing the resolution for the benefit of the community.
Thanks for your reply. This solution works well in our business scenario, but the API response time is too long. We tested the task in Sweden-Central and East-US2; the longest response time could be up to 120 seconds and the shortest is 50 seconds. Is the model overloaded?
By system messge you mean system message auto-injected by Azure and not by users (as per my examples there's no system message send with the queries)?
Yes, I guess so.