I cannot register "metadata" info in Azure OpenAI Batch API

Question

I cannot register "metadata" info in Azure OpenAI Batch API

TERADA, Tomohiko (寺田智彦) 20

Since a couple days ago, I've not able to register "metadata" info when I create new batch Job.

Is there anyone who have same issue?

I call 'create new batch job' API with my local data in "metadata".

I receive the job has been registered successfully with my "metadata".

However, if I call 'list job batch', my metadata is empty.

It's been working properly until Sep-4.

Regards,

romungi-MSFT 49,096 Reputation points Microsoft Employee Moderator

2024-09-06T14:01:13.71+00:00

@TERADA, Tomohiko (寺田智彦) Are you referring to your deployments of batch endpoints through Azure OpenAI studio or REST API? I have referred the official documentation of Azure OpenAI Batch API and the create batch job does not mention the availability of metadata in the reference, see here for details.

I have also tried the Azure OpenAI studio and see the same behavior where there is no option to add metadata in request body.

I have tried to add the metadata through REST API requests during create but the list or GET batch API does not provide the added metadata in the response, it is always set to null.

I think for the Azure OpenAI implementation metadata option might currently be turned off.

If you have set metadata in previous jobs before 4th Sep, do the GET provide this data now? Thanks!!
TERADA, Tomohiko (寺田智彦) 20 Reputation points

2024-09-06T20:36:17.0166667+00:00

@romungi-MSFT ,

Thank you for your comment.

I'm using REST API

If you have set metadata in previous jobs before 4th Sep, do the GET provide this data now?

Yes, I still can get my metadata which I stored before 4th Sep.

I have referred the official documentation of Azure OpenAI Batch API and the create batch job does not mention the availability of metadata in the reference, see here for details.

The document mentions batch object and there is a metadata property.

TERADA, Tomohiko (寺田智彦) 20

I call this and I get my metadata which I stored before 4th Sep.

job_list = client.batches.list(limit=100) print(job_list)

Batch(id='batch_c04f8856-2d44-45f7-8aa9-7cb6bc1625fa', completion_window='24h', created_at=1724118513, endpoint='/chat/completions', input_file_id='file-bd85d9d92940430daccfb09497cb39c5', object='batch', status='completed', cancelled_at=None, cancelling_at=None, completed_at=1724119083, error_file_id='file-9a4edd76-53f1-4a75-94aa-90ab2fefbbad', errors=None, expired_at=None, expires_at=1724204913, failed_at=None, finalizing_at=1724118982, in_progress_at=1724118670, metadata={'user_email': 'tomohiko.g1s8.terada@xxxx', 'data_file': 'xxxxx.xlsx', 'comment': '', 'temperature': '0.7', 'top_p': '1.0', 'frequency_penalty': '0.0', 'presence_penalty': '0.0'}, output_file_id='file-a1046cb4-569b-4504-85cb-03932f07bb57', request_counts=BatchRequestCounts(completed=504, failed=0, total=504))

romungi-MSFT 49,096 Reputation points Microsoft Employee Moderator

2024-09-09T07:30:54.0733333+00:00

@TERADA, Tomohiko (寺田智彦) Thanks for sharing these details. Let me check internally to find out if this feature is working as expected. Thanks!!
Emily 5 Reputation points

2024-09-12T12:55:38.9966667+00:00
@romungi-MSFT I'd like to add that I've just encountered this exact problem. I can retrieve metadata for old batches, but not new ones.

Reproduction steps:

created_batch = batch_client.batches.create( input_file_id=batch_input_file_id, endpoint="/chat/completions", completion_window="24h", metadata={ "description": "some description", } )

created_batch.metadata contains the description, as expected.

On the very next line, I call:

all_batches = batch_client.batches.list()

all_batches.data[0] is the batch I just created (its input_file_id is equal to batch_input_file_id), but its metadata is None
TERADA, Tomohiko (寺田智彦) 20 Reputation points

2024-09-12T13:43:22.8066667+00:00

Still, I cannot retrieve metadata.
So, I switched to OpenAI (not Azure), and now my program works well.
Why did Microsoft make it so that only Azure OpenAI cannot use metadata...?

Answer accepted by question author

0 additional answers

Your answer

romungi-MSFT 49,096 Reputation points Microsoft Employee Moderator

2024-09-06T14:01:13.71+00:00

@TERADA, Tomohiko (寺田智彦) Are you referring to your deployments of batch endpoints through Azure OpenAI studio or REST API? I have referred the official documentation of Azure OpenAI Batch API and the create batch job does not mention the availability of metadata in the reference, see here for details.

I have also tried the Azure OpenAI studio and see the same behavior where there is no option to add metadata in request body.

I have tried to add the metadata through REST API requests during create but the list or GET batch API does not provide the added metadata in the response, it is always set to null.

I think for the Azure OpenAI implementation metadata option might currently be turned off.

If you have set metadata in previous jobs before 4th Sep, do the GET provide this data now? Thanks!!
TERADA, Tomohiko (寺田智彦) 20 Reputation points

2024-09-06T20:36:17.0166667+00:00

@romungi-MSFT ,

Thank you for your comment.

I'm using REST API

If you have set metadata in previous jobs before 4th Sep, do the GET provide this data now?

Yes, I still can get my metadata which I stored before 4th Sep.

I have referred the official documentation of Azure OpenAI Batch API and the create batch job does not mention the availability of metadata in the reference, see here for details.

The document mentions batch object and there is a metadata property.
TERADA, Tomohiko (寺田智彦) 20 Reputation points

2024-09-06T20:50:08.5566667+00:00

I call this and I get my metadata which I stored before 4th Sep.

job_list = client.batches.list(limit=100) print(job_list)

Batch(id='batch_c04f8856-2d44-45f7-8aa9-7cb6bc1625fa', completion_window='24h', created_at=1724118513, endpoint='/chat/completions', input_file_id='file-bd85d9d92940430daccfb09497cb39c5', object='batch', status='completed', cancelled_at=None, cancelling_at=None, completed_at=1724119083, error_file_id='file-9a4edd76-53f1-4a75-94aa-90ab2fefbbad', errors=None, expired_at=None, expires_at=1724204913, failed_at=None, finalizing_at=1724118982, in_progress_at=1724118670, metadata={'user_email': 'tomohiko.g1s8.terada@xxxx', 'data_file': 'xxxxx.xlsx', 'comment': '', 'temperature': '0.7', 'top_p': '1.0', 'frequency_penalty': '0.0', 'presence_penalty': '0.0'}, output_file_id='file-a1046cb4-569b-4504-85cb-03932f07bb57', request_counts=BatchRequestCounts(completed=504, failed=0, total=504))
romungi-MSFT 49,096 Reputation points Microsoft Employee Moderator

2024-09-09T07:30:54.0733333+00:00

@TERADA, Tomohiko (寺田智彦) Thanks for sharing these details. Let me check internally to find out if this feature is working as expected. Thanks!!
Emily 5 Reputation points

2024-09-12T12:55:38.9966667+00:00

@romungi-MSFT I'd like to add that I've just encountered this exact problem. I can retrieve metadata for old batches, but not new ones.

Reproduction steps:

created_batch = batch_client.batches.create( input_file_id=batch_input_file_id, endpoint="/chat/completions", completion_window="24h", metadata={ "description": "some description", } )

created_batch.metadata contains the description, as expected.

On the very next line, I call:

all_batches = batch_client.batches.list()

all_batches.data[0] is the batch I just created (its input_file_id is equal to batch_input_file_id), but its metadata is None
TERADA, Tomohiko (寺田智彦) 20 Reputation points

2024-09-12T13:43:22.8066667+00:00

Still, I cannot retrieve metadata.
So, I switched to OpenAI (not Azure), and now my program works well.
Why did Microsoft make it so that only Azure OpenAI cannot use metadata...?

Answer 1

romungi-MSFT 49,096 Microsoft Employee Moderator

@TERADA, Tomohiko (寺田智彦) I can confirm that there has been a bug with batch jobs that caused the metadata property to be null. A fix has been deployed now and I can see metadata property returns with the data added in the create job. Please see a response of my batch job that was submitted today which contains the metadata.

{
    "cancelled_at": null,
    "cancelling_at": null,
    "completed_at": 1726573123,
    "completion_window": "24h",
    "created_at": 1726572674,
    "error_file_id": "file-4ec49c3f-10e2-429d-aefd-f5b2e38998c5",
    "expired_at": null,
    "expires_at": 1726659073,
    "failed_at": null,
    "finalizing_at": 1726573054,
    "id": "batch_033b4cc2-d00c-4df9-967e-fccde36e6578",
    "in_progress_at": 1726572839,
    "input_file_id": "file-380cd850c29947b695bca88a7006bcae",
    "errors": null,
    "metadata": {
        "modified": "true",
        "user": "abc123"
    },
    "object": "batch",
    "output_file_id": "file-bbb36a01-497e-42c8-88a7-b2e68ecef292",
    "request_counts": {
        "total": 3,
        "completed": 3,
        "failed": 0
    },
    "status": "completed",
    "endpoint": "/chat/completions"
}

Could you please retry with a new job and check if it is working as expected for you. Thanks!!

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

TERADA, Tomohiko (寺田智彦) 20 Reputation points

2024-09-17T13:27:19.09+00:00

@romungi-MSFT ,

Thank you for your support. Now, it works.
TERADA, Tomohiko (寺田智彦) 20 Reputation points

2024-09-25T03:02:13.02+00:00

@romungi-MSFT

Thank you for your support.

Since you've fixed this problem, it's worked.

However, from today, I get the following error. I didn't change my program and my program works with OpenAI API (not Azure).

It looks MS-Azure batch does not accept '' value in metadata like this.

'metadata': {

'param1' : ''

}

If I put some value, it works.

Did you change this ?

==

{'error': {'code': 'UserError', 'severity': None, 'message': 'The CreateBatchJobRequest request provided has the following validation errors. Please resolve them and try again.\nErrors:\n[\n {\n "PropertyName": "Metadata.Values[2]",\n "ErrorMessage": "Metadata values must not be null or empty.",\n "AttemptedValue": "",\n "ErrorCode": "PredicateValidator"\n }
romungi-MSFT 49,096 Reputation points Microsoft Employee Moderator

2024-09-25T04:51:02.95+00:00

@TERADA, Tomohiko (寺田智彦) I will try to pass this message to the concerned team, but I think the above message is a validation that is in place for null metadata.
TERADA, Tomohiko (寺田智彦) 20 Reputation points

2024-09-25T05:00:31.0433333+00:00

@romungi-MSFT ,

Thank you for your response.

I think the above message is a validation that is in place for null metadata.

They say "Metadata.Value[2]", it means 3rd item of metadata is null. and if I put non-null value in the 3rd item, it works.
TERADA, Tomohiko (寺田智彦) 20 Reputation points

2024-10-07T16:11:07.3266667+00:00

@romungi-MSFT ,

It looks it happen again, now I cannot get my metadata.

Can you please check?

romungi-MSFT 49,096 Microsoft Employee Moderator

@TERADA, Tomohiko (寺田智彦) i just tried again and it seems to show my metadata after completion too.

{
    "cancelled_at": null,
    "cancelling_at": null,
    "completed_at": 1728562713,
    "completion_window": "24h",
    "created_at": 1728562116,
    "error_file_id": "file-981ba4fa-7ea7-4a6b-a700-ec2ea1fab233",
    "expired_at": null,
    "expires_at": 1728648515,
    "failed_at": null,
    "finalizing_at": 1728562637,
    "id": "batch_2572739f-db95-4f49-a26b-4514a3d9e8b4",
    "in_progress_at": 1728562490,
    "input_file_id": "file-380cd850c29947b695bca88a7006bcae",
    "errors": null,
    "metadata": {
        "modified": "true",
        "user": "abc12356"
    },
    "object": "batch",
    "output_file_id": "file-b1d55954-f4c0-4bc7-9a6f-015764f8ded3",
    "request_counts": {
        "total": 3,
        "completed": 3,
        "failed": 0
    },
    "status": "completed",
    "endpoint": "/chat/completions"
}

Share via

I cannot register "metadata" info in Azure OpenAI Batch API

0 additional answers

Your answer