We've been asking for Graph API endpoints for Exchange management for years now, to no avail. There are some operations exposed via REST endpoints, as you mentioned below, however calling those outside of the PowerShell module is not officially supported. That said, if you don't care about the "supported" bit, you can certainly create a solution with no PowerShell dependencies.
Exchange Online Management Rest API
Are there any plans to move Exchange Online Management of Mailboxes, Global Contacts, and Distribution Lists to a REST API?
I understand that there is a PowerShell module called ExchangeOnlineManagement, which is now completely backed by REST APIs. When can we call these APIs with other programming languages such as Python.
Currently, we WINRM with python to a windows server that runs powershell, however, this is a very inefficient process and actually opens the code up to vulnerabilities.
I would expect that Exchange Online would follow the same principals as Azure AD objects as they can be created using Graph.