Have you seen this?
https://o365blog.com/post/limit-user-access/
Related: Also, you can use a CA for blocking access to the mgmt tools except Azure AD PS:
https://learn.microsoft.com/en-us/azure/role-based-access-control/conditional-access-azure-management