@Leif Johansson
If you are NTFS permission on the servers then the end user computers should be joined to Azure AD domain in order to have access to the file share on the file server.
There is a well explained video, please go through this [article.][1]
When using Azure AD for the authentication only 3 NTFS permission types can be granted:
Storage File Data SMB Share Reader allows read access in Azure Storage file shares over SMB.
Storage File Data SMB Share Contributor allows read, write, and delete access in Azure Storage file shares over SMB.
Storage File Data SMB Share Elevated Contributor allows read, write, delete and modify NTFS permissions in Azure Storage file shares over SMB.
https://learn.microsoft.com/en-us/azure/storage/files/storage-files-identity-auth-active-directory-enable#2-assign-access-permissions-to-an-identity
Azure File Share level is using RBAC access.While for root,directory and file level uses NTFS. Do you have link on how to setup the NTFS permission? Here how to set up permission at folder level on the file share: https://learn.microsoft.com/en-us/azure/storage/files/storage-files-identity-auth-active-directory-domain-service-enable#configure-ntfs-permissions-with-windows-file-explorer
Azure File Sync service contains Sync Groups each Sync group has a File Share acting as a cloud endpoint and can contains several servers acting as a Server Endpoints (cache) so in this case the end users should access the file shares on the file servers (server endpoints). Remember that Azure File sync and Azure files are related products but with different goals. If you only have a Azure File Share then end users should map be file share directly as any other file share.
strong text[1]: https://azure.microsoft.com/en-in/blog/preview-of-active-directory-for-authentication-on-azure-file/
Hope this helps!
Kindly let us know if the above helps or you need further assistance on this issue.