Apologies for delayed response on this.
At the moment, there is no workaround. You need to add all IP address to the allow-list.
Hope this helps.
Please 'Accept as answer' if the provided information is helpful, so that it can help others in the community looking for help on similar topics.