"ErrorDataSourceOperationFailed" when getting a contact by id

Question

Calling API v1.0/me/contacts/{id}
works for most contacts in my test account, but for quite a few trying to get their details results in:

Code 500
{
    "error": {
        "code": "ErrorDataSourceOperationFailed",
        "message": "There was a problem accessing Active Directory. Check your network connections and try again."
    }
}

• I'm using a fresh access token, which works for other contacts.
• The API repeatedly works fine for most contacts, and repeatedly shows the above error for specific contacts

here's an example:

GET https://graph.microsoft.com/v1.0/me/contacts/AQMkADAwATNiZmYAZC00NThlLTRiMjctMDACLTAwCgBGAAADA5bCwgDA6wdAr08gvjYnWxMHAEQv6Yx2QhFKvZ4NbvDRllUAAAIBDgAAAEQv6Yx2QhFKvZ4NbvDRllUAAAAdH6VAAAAA

=> 500

{
    "error": {
        "code": "ErrorDataSourceOperationFailed",
        "message": "There was a problem accessing Active Directory. Check your network connections and try again."
    }
}

using the same config and token:

GET https://graph.microsoft.com/v1.0/me/contacts/AQMkADAwATNiZmYAZC00NThlLTRiMjctMDACLTAwCgBGAAADA5bCwgDA6wdAr08gvjYnWxMHAEQv6Yx2QhFKvZ4NbvDRllUAAAIBDgAAAEQv6Yx2QhFKvZ4NbvDRllUAAAAdH6UxAAAA
=>

{
    "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#users('my.test.account%40outlook.com')/contacts/$entity",
    "@odata.etag": "W/\"EQAAABYAAABEL+mMdkIRSr2eDW7w0ZZVAAAdJfL9\"",
    "id": "AQMkADAwATNiZmYAZC00NThlLTRiMjctMDACLTAwCgBGAAADA5bCwgDA6wdAr08gvjYnWxMHAEQv6Yx2QhFKvZ4NbvDRllUAAAIBDgAAAEQv6Yx2QhFKvZ4NbvDRllUAAAAdH6UxAAAA",
    ....
}

Also note that listing my contacts using

GET https://graph.microsoft.com/v1.0/me/contacts?$skip=320
will return that problematic contact ID above, and checking OWA I can see this contact exists in the account.

{
    "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#users('my.test.account%40outlook.com')/contacts",
    "value": [
        {
            "@odata.etag": "W/\"EQAAABYAAABEL+mMdkIRSr2eDW7w0ZZVAAAdJelB\"",
            "id": "AQMkADAwATNiZmYAZC00NThlLTRiMjctMDACLTAwCgBGAAADA5bCwgDA6wdAr08gvjYnWxMHAEQv6Yx2QhFKvZ4NbvDRllUAAAIBDgAAAEQv6Yx2QhFKvZ4NbvDRllUAAAAdH6VAAAAA",
            "createdDateTime": "2022-02-08T11:17:25Z",
            "lastModifiedDateTime": "2022-02-08T11:17:25Z",
    ...
}