You should remove the Azure AD join, remove the device, and turn off automatic registration.
You can remove the Azure AD join by running dsregcmd /status
.
Then, if you know the object ID of the device you can try removing the device with this command:
Remove-AzureADDevice -ObjectId "deviceIDhere"
See also: How do I remove an Azure AD registered state for a device locally?
As mentioned in the documentation, deleting an Azure AD device does not remove registration on the client. It will only prevent access to resources using device as an identity (e.g Conditional Access). When a user is deleted or disabled in Azure AD, it's not immediately known to the Windows device. So users who signed in previously can access the desktop with the cached username and password, typically for ~4 hours after deletion.
I would unjoin the device, remove the device, remove the user from the organization, and permanently delete the user.