The user is not supposed to call the azure login endpoint. Your application is supposed to call the endpoint.
The user calls an application endpoint passing username and password and maybe scopes. The application calls the graph api and returns the token, which should be usable to call the api.
the password flow is used when the user can not use a browser to login, but wants to login to the app directly with their username password.
You should just expose the endpoint, not your clientid and secret.