@Jon Z Thank you for reaching out.
If your Azure MySQL database is publicly accessible, you should be able to connect to it from Power BI without using a data gateway. However, if your database is secured within a VNet or uses private endpoints, a data gateway might be necessary.
Please check the following documents for furthermore details.
https://learn.microsoft.com/en-us/azure/mysql/flexible-server/connect-with-powerbi-desktop
More questions? Try the Power BI community
Please do also comment below if you need more information.
Regards,
Oury