I might be missing the point here, but isn't that why we have the Delegate permissions model? Here's a simple example - me granting user-level permissions for the Graph explorer:
The permissions will then be reflected on the corresponding app, just for the user in question. If any other users needs such permissions, another consent is needed. And yes, you will have to use an admin account to consent to each of the users individually, but it's doable. You usually address this via the prompt=admin_consent query parameter: https://learn.microsoft.com/en-us/azure/active-directory/develop/howto-convert-app-to-be-multi-tenant#admin-consent