Please do the following steps. Using Azure Portal once you select the database, make a click on Set Server Firewall.
Then choose selected networks.
Then you just need to add your current client IP address and any public IP address you want.
You don' need to do more than that.