kshepic Thanks for posting your question in Microsoft Q&A. Unfortunately, there is no option to remove the custom widget from the UI and we have a backlog item to implement this feature. Meanwhile, you can create a support ticket with azure support team to manually delete the widget. If you don't have a support plan, let me know and I will assist you with this issue.
Sorry for the inconvenience caused. I will also pass the feedback to our product team internally. Let me know if you have any questions.