Hi @Gerrit
You can use Conditional Access Policies to define locations (specific Public IPv4 addresses such as your offices/vpn):
Define locations
Block access by location with Azure AD Conditional Access
https://learn.microsoft.com/en-us/power-platform/admin/restrict-access-online-trusted-ip-rules
-------------------------------
If this is helpful please accept answer.