Thanks @Ryan Hill ,
In my case I have to perform the server-based flow (one configured user) to a customer's Sharepoint Site via Microsoft Graph. How about I use ROPC flow. Here I can provide detailed scopes like https://graph.microsoft.com/User.Read, https://graph.microsoft.com/Sites.Read to fetch the access token.
Wondering whether the use of ROPC would allow me to have more fine level access as there as I define a user on the token fetch. Will Sharepoint user-level access (sites, documents) come into play (or will it just be the high-level app-based access/permission levels)?