Hi Wenbo Peng •,
Welcome to Microsoft Q&A forum.
As I understand, you have an Azure Database for MySQL Flexible Server which you can connect via MySQL workbench but not via PowerShell.
Could you please let us know if you are using Azure Cloud Shell for this?
If your Azure Database for MySQL - Flexible Server has to connect with private access connectivity method, follow https://learn.microsoft.com/en-us/azure/mysql/flexible-server/quickstart-create-connect-server-vnet
Refer to this blogpost MySQL Flexible Server PowerShell Module:
Hope this helps. Let us know if your ask is different.
Thanks