Create the relationship and scope it to a specific group with just those users
https://learn.microsoft.com/en-us/exchange/sharing/organization-relationships/create-an-organization-relationship
To set which users will share calendar free/busy information, select one of the following values:
Everyone in your organization
A specified security group