You won't be able to change it from Windows to Azure AD authentication, as SSRS does not support Azure AD authentication. But what you can do is use the Azure Active Directory Application Proxy to connect to the Power BI Report Server and SQL Server Reporting Services (SSRS) 2016 or later. See Configure Power BI Report Server with Azure Application Proxy
Another way to make this work is to map your Azure AD identities to Windows authentication, as described here.
Let me know if this helps!
See also:
Modernizing SSRS
SSRS and Azure SQL
How to setup SSRS with an Azure SQL Server Database