But restore defaults from the ... menu just brought them back.
They do not appear in Home as various MS blog suggest they do (files that are favourites do) and they appear after Home, gallery (new in preview), my personal OneDrive, my 5 business OneDrives, desktop, downloads which is bit annoying (I like pinned at the top).
I also notice that my desktop appears pined (with pin icon) and I can't remove it using unpin (the menu item appears but the pin never goes).