@Vidar Welcome to Microsoft Q&A, thank you for posting your here!!
Renaming functionality is only available in Data Lake Gen 2 Storage (Hierarchical Namespace)
I assume you might be checking in GPV2 storage or Blob storage as we don't have folder or file rename option in GPV2 Storage account.
For GPV2 Storage account with Hierarchal Namespace enabled.
For GPV2 Storage account with Hierarchal Namespace disabled.
Hope this helps!
Kindly let us know if the above helps or you need further assistance on this issue.
Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helpsyou, this can be beneficial to other community members.