Graph API - Returning 404 in certain data centers when accessing on-prem mailboxes

I recently created a process that reads/writes contacts in users' mailboxes on-prem via Graph. This process works as expected most of the time but certain times it fails with a 404. I've narrowed it down to the datacenter Graph is using. I've verified nothing is causing this issue at the network perimeter and no API calls are actually reaching the Exchange servers when the 404 is returned.
404 - North Central US
DEBUG: GET https://graph.microsoft.com/v1.0/users/zHM.Test%40fomain.com/contacts?$top=100
HTTP/1.1 404 Not Found
request-id: f245c3b7-4852-4494-aaf9-1424bfd9e41c
client-request-id: f245c3b7-4852-4494-aaf9-1424bfd9e41c
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"North Central
US","Slice":"SliceC","Ring":"2","ScaleUnit":"000","RoleInstance":"AGSFE_IN_56"}}
Strict-Transport-Security: max-age=31536000
Cache-Control: private
200 - East US
DEBUG: GET https://graph.microsoft.com/v1.0/users/zHM.Test%40domain.com/contacts?$top=100
HTTP/1.1 200 OK
Vary: Accept-Encoding
request-id: 2d014679-7c0c-4a68-bb3a-26e7cf91af53
client-request-id: 2d014679-7c0c-4a68-bb3a-26e7cf91af53
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"East
US","Slice":"SliceC","Ring":"5","ScaleUnit":"003","RoleInstance":"AGSFE_IN_41"}}
OData-Version: 4.0
Strict-Transport-Security: max-age=31536000
Cache-Control: private
Date: Wed, 25 Nov 2020 20:31:56 GMT
404 - North Europe
DEBUG: GET https://graph.microsoft.com/v1.0/users/zHM.Test%40domain.com/contacts?$top=100
HTTP/1.1 404 Not Found
request-id: 4ddb30db-e9f7-4a20-bd46-d58d9226fc0b
client-request-id: 4ddb30db-e9f7-4a20-bd46-d58d9226fc0b
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"North
Europe","Slice":"SliceC","Ring":"4","ScaleUnit":"005","RoleInstance":"AGSFE_IN_2"}}
Strict-Transport-Security: max-age=31536000
Cache-Control: private
Date: Wed, 25 Nov 2020 21:52:06 GMT
Without pagination I still receive the error.
I can consistently repro the issue when initiating from our London location since that hits the 'North Europe' datacenter for Graph. Like I said originally, this is consistent with certain datacenters.
I haven't done it in Graph Explorer yet, but it should happen there to if Graph Explorer uses one of the problematic datacenters.
Thanks for the update. Please note that you're trying to use Graph API with Exchange Hybrid deployment and it's in preview.
Case has been logged. I referenced you and the stackoverflow thread. They are moving forward with the ticket so far.
Sign in to comment