Troubleshoot NFS Azure file shares
Note
CentOS referenced in this article is a Linux distribution and will reach End Of Life (EOL). Consider your use and plan accordingly. For more information, see CentOS End Of Life guidance.
This article lists common issues related to NFS Azure file shares and provides potential causes and workarounds.
Important
The content of this article only applies to NFS shares. To troubleshoot SMB issues in Linux, see Troubleshoot Azure Files problems in Linux (SMB). NFS Azure file shares aren't supported for Windows.
Applies to
File share type | SMB | NFS |
---|---|---|
Standard file shares (GPv2), LRS/ZRS | ||
Standard file shares (GPv2), GRS/GZRS | ||
Premium file shares (FileStorage), LRS/ZRS |
chgrp "filename" failed: Invalid argument (22)
Cause 1: idmapping isn't disabled
Because Azure Files disallows alphanumeric UID/GID, you must disable idmapping.
Cause 2: idmapping was disabled, but got re-enabled after encountering bad file/dir name
Even if you correctly disable idmapping, it can be automatically re-enabled in some cases. For example, when Azure Files encounters a bad file name, it sends back an error. Upon seeing this error code, an NFS 4.1 Linux client decides to re-enable idmapping, and sends future requests with alphanumeric UID/GID. For a list of unsupported characters on Azure Files, see this article. Colon is one of the unsupported characters.
Workaround
Make sure you've disabled idmapping and that nothing is re-enabling it. Then perform the following steps:
Unmount the share.
Disable idmapping with:
sudo echo Y > /sys/module/nfs/parameters/nfs4_disable_idmapping
Mount the share back.
If running rsync, run rsync with the
-numeric-ids
argument from a directory that doesn't have a bad directory or file name.
Unable to create an NFS share
Cause: Unsupported storage account settings
NFS is only available on storage accounts with the following configuration:
- Tier: Premium
- Account Kind: FileStorage
Solution
Follow the instructions in Create an NFS file share.
Can't connect to or mount an NFS Azure file share
Cause 1: Request originates from a client in an untrusted network/untrusted IP
Unlike SMB, NFS doesn't have user-based authentication. The authentication for a share is based on your network security rule configuration. To ensure that clients only establish secure connections to your NFS share, you must use either the service endpoint or private endpoints. To access shares from on-premises in addition to private endpoints, you must set up a VPN or ExpressRoute connection. IPs added to the storage account's allowlist for the firewall are ignored. You must use one of the following methods to set up access to an NFS share:
-
Accessed by the public endpoint.
Only available in the same region.
You can't use VNet peering for share access.
You must add each virtual network or subnet individually to the allowlist.
For on-premises access, you can use service endpoints with ExpressRoute, point-to-site, and site-to-site VPNs. We recommend using a private endpoint because it's more secure.
The following diagram depicts connectivity using public endpoints:
-
Access is more secure than the service endpoint.
Access to NFS share via private link is available from within and outside the storage account's Azure region (cross-region, on-premises).
Virtual network peering with virtual networks hosted in the private endpoint give the NFS share access to the clients in peered virtual networks.
You can use private endpoints with ExpressRoute, point-to-site VPNs, and site-to-site VPNs.
Cause 2: Secure transfer required is enabled
NFS Azure file shares don't currently support double encryption. Azure provides a layer of encryption for all data in transit between Azure datacenters using MACSec. You can only access NFS shares from trusted virtual networks and over VPN tunnels. No extra transport layer encryption is available on NFS shares.
Solution
Disable Secure transfer required in your storage account's configuration blade.
Cause 3: nfs-utils, nfs-client or nfs-common package isn't installed
Before running the mount
command, install the nfs-utils, nfs-client, or nfs-common package.
To check if the NFS package is installed, run:
The same commands in this section apply to CentOS and Oracle Linux.
sudo rpm -qa | grep nfs-utils
Solution
If the package isn't installed, install the package using your distro-specific command.
The same commands in this section apply to CentOS and Oracle Linux.
OS Version 7.X
sudo yum install nfs-utils
OS Version 8.X or 9.X
sudo dnf install nfs-utils
Cause 4: Firewall blocking port 2049
The NFS protocol communicates to its server over port 2049. Make sure that this port is open to the storage account (the NFS server).
Solution
Verify that port 2049 is open on your client by running the following command. If the port isn't open, open it.
sudo nc -zv <storageaccountnamehere>.file.core.windows.net 2049
Cause 5: Storage account deleted
If you're unable to mount the file share due to error: connection timed out, the storage account containing the file share might be deleted accidentally.
Solution
Recover the storage account. Then, delete and re-create the private endpoint so it's associated with the new storage account resource ID.
ls hangs for large directory enumeration on some kernels
Cause: A bug was introduced in Linux kernel v5.11 and was fixed in v5.12.5
Some kernel versions have a bug that causes directory listings to result in an endless READDIR sequence. Small directories where all entries can be shipped in one call don't have this problem. The bug was introduced in Linux kernel v5.11 and was fixed in v5.12.5. So anything in between has the bug. RHEL 8.4 has this kernel version.
Workaround: Downgrade or upgrade the kernel
Downgrading or upgrading the kernel to anything outside the affected kernel should resolve the issue.
System commands fail with the "File not found" error
Cause
Linux 32-bit applications that rely on inode numbers might not work as expected with Azure Files due to the formatting of the 64-bit inode numbers generated by the NFS service.
Solution
To resolve this issue, use one of the following methods:
Compress the 64-bit inode numbers to 32 bits by using the
nfs.enable_ino64=0
kernel boot option.Set the module parameter by adding
options nfs enable_ino64=0
to the /etc/modprobe.d/nfs.conf file and rebooting the VM.
You can also persist this kernel boot option in the grub.conf file. For more information, see the documentation for your Linux distribution.
Unable to change the ownership of files and directories
Cause
Permissions on NFS file shares are enforced by the client OS rather than the Azure Files service. If the Root Squash setting is enabled on an NFS file share, the root user on the client system is treated as an anonymous (non-privileged) user for access control purposes. This means that even if you're logged in as root on the client system, you can't use the chown
command to change the ownership of files and directories that you don't own.
Solution
In the Azure portal, navigate to the file share and select Properties. Change the Root Squash setting to No Root Squash. For more information, see Configure root squash for Azure Files.
With No Root Squash enabled, the root user on the client system has the same privileges as the root user on the server system. You can now use chown
to change the ownership of any file or directory in the share, regardless of the current owner. After you make the changes, you can re-enable Root Squash if necessary.
Need help?
If you still need help, contact support to get your problem resolved quickly.
See also
- Troubleshoot Azure Files
- Troubleshoot Azure Files performance
- Troubleshoot Azure Files connectivity (SMB)
- Troubleshoot Azure Files authentication and authorization (SMB)
- Troubleshoot Azure Files general SMB issues on Linux
Contact us for help
If you have questions or need help, create a support request, or ask Azure community support. You can also submit product feedback to Azure feedback community.