You can´t. Addins are Office/machine specific not users specific. Redirect them to a different RDS. Or install it for all and disable it by Registry per user.
change LoadBehavior:
https://learn.microsoft.com/en-us/visualstudio/vsto/registry-entries-for-vsto-add-ins?view=vs-2019