Hello David Shearing,
Thank you for posting this case in Microsoft Community!
Based on your description: Autocomplete keeps reappearing in Outlook. I would like to cooperate with you working on this case. Since you have performed all the troubleshooting steps from your side, it is suggested that you may also check following steps if could help you further:
- Firstly, you may open this account contact in Outlook and click the three dots at the top-right corner under the contact's name, then choose Link Contacts from the drop-down list.

- In the dialog box that pops up, you'll see a list of contacts that are already linked. If there is showing the linked contacts for the old account, you may unlink and check again.

- Next, you may let your Microsoft 365 admin to connect Exchange Online Power Shell to run the script step by step with admin level. You need to check if the old contact is still showing in public contact list under your tenant Get-Contact. If Yes, you may let them help to remove it completely. (Note: If you are not sure who is admin, you may check this Microsoft article: How do I find my Microsoft 365 admin? - Microsoft Support)
Script:
Install-Module -Name ExchangeOnlineManagement -allowprerelease
Import-module ExchangeOnlineManagement
Connect-ExchangeOnline -UserPrincipalName [Microsoft 365 admin account] -ShowProgress $true
Get-Contact

If above workaround not helpful, please do not mind that we could not provide further assistance because need to check from backend side and we appreciate your efforts in this case. For my suggestion, you may contact Microsoft Technical Support Team with a ticket opened: Ways to contact support for business products - Admin help. Go to Office365 admin center>Support>New Service Request. The support team over there has higher permission than us, they can remotely help you and collect logs at your convenience. They can also involve more resources to investigate the issue from backend side. This would be the most efficient way in handling this case for you.
Your understanding and patience will be appreciated.
Thank you!