Hi @ChristopherCoates Thank you for reaching out to us.
As I understand you are delete a device from Azure AD which is tagged as Windows Autopilot device.
Researched on your issue and found this article https://www.prajwaldesai.com/delete-windows-autopilot-device-from-azure-ad/#:~:text=1%20First%20login%20to%20Microsoft%20Endpoint%20Admin%20centre,take%20a%20few%20minutes%20to%20complete.%20More%20 which has the steps to delete the device from Azure AD.
Let me know if this article helps to resolve your issue or not.
@ChristopherCoates
Thank you for the sharing the steps/script which you followed to resolved the issue.
Please remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution.