This is currently a product limitation like you observed, but I've reached out the product group to see if there is a better workaround than the ones you mentioned or plans to improve this functionality. Like you said, Azure AD does not from authorization requests to non-HTTPS URLs, but http followed by localhost is the only exception. The https:
scheme is required for any external domain name.
You can use your current workaround of serving your localhost URLs via https, or you can use a different domain name for your development environment that supports https.
As a security measure, Azure AD requires that redirect URIs use a secure scheme to prevent attacks.
I've reached out to the product team though to share your feedback and will update this thread accordingly if they are able to offer some better workarounds and updates on this functionality.
Switching to a different OAuth2 provider may work, but it may not have all of the security options offered by Azure AD.