Unfortunately not, the rights are tied to the entire Azure File Share, so everything that the user has access to they can see.
Azure Storage explorer, will still display the Tables etc, but if the user doesn't have access to see them, then they can't drill down.
If it's only specific folders or files, it may be better to move them to a separate file share that the user has access to only.