@Jani Hämäläinen Thank you for reaching out.
Azure SQL Server does support Power Apps SQL Server connections using Microsoft Entra ID Integrated authentication. The SQL Server connector IN Microsoft Power app allows you to use Microsoft Entra Integrated, SQL Server Authentication, and Windows Authentication.
You mentioned when i have created SQL Server connection in Power apps with Microsoft Entra ID Integrated as authentication type i get error: "Server doesn't exist". When tested with allowing all ip addresses it connected to server and databases.
Since the connection works when all IP addresses are allowed, ensure that the IP address of the Power Apps service is included in the allowed IP addresses for your Azure SQL Server.
The default for the Public network access setting is Disable. Customers can choose to connect to a database by using either public endpoints (with IP-based server-level firewall rules or with virtual-network firewall rules), or private endpoints (by using Azure Private Link), as outlined in the network access overview.
When Public network access is set to Disable, only connections from private endpoints are allowed. All connections from public endpoints will be denied.
https://learn.microsoft.com/en-us/power-apps/maker/canvas-apps/connections/sql-server-security
I would suggest to also post your queries in the Microsoft Power App community to get more insights from other experts. https://community.powerplatform.com/