Has anything changed with this behavior in the last 6-8 months or so? I encountered this same issue way back - RDSH server deleted before being properly removed from the RDS configuration, thus being orphaned and unable to be removed using the UI. Any attempts to remove them would result in an error and they would not 'go away'. I had to use the linked article, or one like it, to remove orphaned RDSH servers back then. I just had a few servers that were recently orphaned (The VMs were deleted), and for whatever reason (Maybe I'm just crazy lol), I tried removing them using the UI, and I got a 'completed with errors' removing the RDSH from the collection it was assigned to, I was then able to remove the collection, and then I went to 'RDS > Servers' and removed the server there, got an error, but... It's gone (Before, it would error and the machine would never go away). I pulled up SSMS and queried the DB to see what it contained, and the server doesn't appear to exist in the database any more. So it seems either this is some sort of cosmic fluke, or something's changed, which if something's changed, that would be great, as I can't imagine a reason to not be able to delete an orphaned server from the UI, with some sort of message like 'make sure this server is offline permanently' similar to how you can delete a dead DC from AD from within ADUC since 2003 or 2008 (don't recall exactly when that became possible), prior to that, to delete a dead DC was messy as you had to get into ADSIEdit and such.