Hello @Simon Arndt ,
Welcome to Microsoft Q&A Platform. Thank you for reaching out & hope you are doing well.
I understand that you would like to know if there is a way to configure access based on Azure AD Groups in Azure point to site VPN, so that it will allow access to a specific server only for a specific user group.
Yes, there are currently 2 options available to achieve your requirement.
First option:
If you are using normal VPN gateway, then you can configure P2S for access based on users and groups via Azure AD authentication.
When you use Azure AD as the authentication method for P2S, you can configure P2S to allow different access for different users and groups. If you want different sets of users to be able to connect to different VPN gateways, you can register multiple apps in AD and link them to different VPN gateways.
Refer: https://learn.microsoft.com/en-us/azure/vpn-gateway/openvpn-azure-ad-tenant-multi-app
But this will require different VPN gateways for different sets of users.
Second option:
You can configure user groups and IP address pools for P2S User VPNs within Azure Virtual WAN which is in preview.
If you use Azure Virtual WAN, you can configure P2S User VPNs to assign users IP addresses from specific address pools based on their identity or authentication credentials by creating User Groups.
A User Group or policy group is a logical representation of a group of users that should be assigned IP addresses from the same address pool.
Gateways using Azure Active Directory authentication can use Azure Active Directory Group Object IDs to determine which user group a user belongs to. If a user is part of multiple Azure Active Directory groups, they're considered to be part of the Virtual WAN user group that has the lowest numerical priority.
Refer: https://learn.microsoft.com/en-us/azure/virtual-wan/user-groups-about
https://learn.microsoft.com/en-us/azure/virtual-wan/user-groups-create
Kindly let us know if the above helps or you need further assistance on this issue.
Please "Accept the answer" if the information helped you. This will help us and others in the community as well.