Hi @Stijn Mohr
I'd recommend raising a ticket with Azure Billing:
https://azure.microsoft.com/en-us/support/create-ticket
Deleting the Purview account should stop billing, but you can inquire for further details.
-----------------------------------
If this is helpful please accept answer.