Hi Robbo,
Please provide the following information for our narrowing down this issue:
1.Try different browsers.
2.Try using another SharePoint administrator account
3.Is the column created as a list column in list settings or created as a site column by going to site settings -> site columns?
4.Does the issue happen to different column types? Which kind of column type do you use?
5.Does the issue happen to this specific list? How about creating a new list in this affected site collection? Is the issue happening to other site collections?
6.Are you using SharePoint designer 2013? If so, please check if the column can be deleted in SharePoint designer as below. If you can't do that, please also share with us a result.
Regards,
Sky