@Vukotic, Branislav
Thank you for your post and I apologize for the delayed response!
From the documentation you provided, I didn't see any other ports that would need to be open when you enable the Key Vault's Firewall, since your App Registration will be accessing the Key Vault via the application-only authentication option, you shouldn't have any issues with your Azure AD App.
As you mentioned, after enabling your Key Vault's Firewall I'd make sure you don't block traffic to Port 443 or 80 since all traffic to a key vault for all three functions (authentication, management, and data plane access) goes over HTTPS/ HTTP (occasionally). Also, if your firewall supports only IP address ranges you can add the Microsoft Azure Datacenter IP Ranges and enable the Allow trusted services option.
Troubleshooting Firewall Issues:
Because the Key Vault service uses other Azure resources like PaaS, it's not possible to provide a specific range of IP addresses that the Key Vault service endpoints will have at any particular time. However, if you're having issues after enabling your KV Firewall, you can troubleshoot Firewall issues using your browser's Developer Tool (F12) or you can Capture a Fiddler Trace.
Once you figure out what IP is being blocked, you can then add it your IPv4 addresses as 12.345.678.901
or 12.345.678.0/24
Additional Links:
Firewall Settings - Different ways that the Azure Key Vault firewall can be configured
Key Vault virtual machine extension for Windows - The Key Vault VM extension provides automatic refresh of certificates stored in an Azure key vault.
The Key Vault request operation flow with authentication - Key Vault Authentication flow example.
If you have any other questions, please let me know.
Thank you for your time and patience throughout this issue.
----------
Please remember to "Accept Answer" if any answer/reply helped, so that others in the community facing similar issues can easily find the solution.