@Sandra Georgine Marquardt I am glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this!
Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to "Accept " the answer. Accepted answers show up at the top, resulting in improved discoverability for others.
Issue: Cannot delete blob using Storage Explorer or Azure Portal for a Storage Account (StorageV2) with Private Endpoints and disabled public access
Cause & Solution: Customer shared - Turns out that it was all due to the additional step of the Storage Explorer to query GetFileProperties. This tries to access data in Data Lake Storage Gen2 (using privatelink.dfs.core.windows.net). Since we had only set a private endpoint for Blob service, i.e. privatelink.blob.core.windows.net, we couldn't delete the blobs. Adding an additional private endpoint for Data Lake Storage Gen2 resolved the issue.
If your issue remains unresolved or have further questions, please let us know in the comments how we can assist. We are here to help you and strive to make your experience better and greatly value your feedback.