I was able to figure it out from other source, but I am posting the answer here in case somebody else finds it helpful:
So there is no direct API to add users, but there is this API you can find below which adds role assignment
https://learn.microsoft.com/en-us/rest/api/authorization/role-assignments/create-by-id#create-role-assignment-by-id
Using the given API we can assign this role '1d18fff3-a72a-46b5-b4a9-0b38a3cd7e63' and provide the user principal id (we can get user details from graph API)
(Ref. https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles).
To conclude this role assignment is equivalent to adding user to the app group.