This error occurs if the server is offline or doesn't have network connectivity. Since the server is no longer available, unregister the server in the portal, which will delete the server endpoints. To delete the server endpoints, follow the steps that are described in Unregister a server with Azure File Sync.
Hope this helps! If you are still facing issues deleting them please let me know and we can work with you directly.
-------------------------------
Please don’t forget to "Accept the answer" and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.