Here may be a solution for you:
Login Microsoft admin center, then create a filter with for A1 license: it will find all mailboxes which assign with A3 license
Then you could click the Checkbox like below to choose all mailbox, then replace for those mailboxes:
If you want to use PowerShell to do this one, I will help you move this thread to dev forum.
If the answer is helpful, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.