401 InvalidAuthenticationToken: Access token has expired or is not yet valid.

We are using Microsoft Graph API to send Outlook emails.
The access token expires after ~1 hour, we refresh it with the refresh token and it works just fine.
However, sometimes, the access token is refreshed successfully, we see the new expiration date which is 1 hour in the future, but trying to use the newly obtained access token we get the following error:
401 InvalidAuthenticationToken: Access token has expired or is not yet valid. from
This doesn't happen every time we refresh the token, sometimes it works well for several months and then we get the above error.
the error says has expired or is not yet valid, it's not expired because the expiration date is 1 hour in the future by the time we see the error as it's just refreshed.
so we think it might be or is not yet valid
, do we have to wait for some time before we start using the new access token, Any other suggestions?