Issues Inserting Data into ClickHouse on Azure Container

Question

Issues Inserting Data into ClickHouse on Azure Container

Sid 65

Experiencing a problem similar to the one described in this GitHub issue while trying to host ClickHouse on an Azure container. It seems related to the CIFS file system being used. Is there a way to format the file system to ext4? Additionally, what are the recommended approaches for hosting ClickHouse on Azure, aside from using ClickHouse Cloud, as I need more location options than currently provided?

Prrudram-MSFT 28,201 Reputation points Moderator

2025-02-01T15:14:07.93+00:00
Hello Sid

Which services on azure are you using exactly to host ClickHouse? Is it Azure Kubernetes service, or container instance or container apps? As for your question, assuming you are using Azure Container Instances (ACI), I think it could be likely due to the limitations of the CIFS (SMB) file system used by Azure File Share, which does not support POSIX permissions and ownership in the same way as ext4. Unfortunately, Azure Container Instances do not natively support mounting file shares with ext4 file systems. Here are some considerations and alternatives:

Addressing the File System Issue:

Formatting to ext4:

Direct formatting of Azure File Share to ext4 is not supported. Azure File Share uses CIFS/SMB, which is fundamentally different from ext4. You cannot change the underlying file system of Azure File Share to ext4 as it's managed by Azure.

Workaround for Permissions: One approach to handle permissions issues is to run ClickHouse with elevated privileges or to adjust the ownership of the mount point inside the container. However, this doesn't solve the underlying file system compatibility problem but might mitigate some permission issues.

Recommended Approaches for Hosting ClickHouse on Azure:

Azure Virtual Machines (VMs):

Control: You have full control over the VM, including the ability to format disks to ext4 or any other file system you prefer.

Setup: Deploy ClickHouse on Azure VMs where you can configure storage with ext4 or XFS for better performance and compatibility.

Scaling: You can scale by adding more VMs and setting up ClickHouse in a distributed configuration for better performance and fault tolerance.

Considerations: This approach requires more management overhead but gives you flexibility in configuration, including file system choices.

Azure Kubernetes Service (AKS):

Container Orchestration: Deploy ClickHouse in containers managed by Kubernetes. Here, you can use Azure Disk Storage, which supports ext4, by attaching persistent volumes formatted with ext4 to your pods.

Scalability: Kubernetes provides dynamic scaling capabilities, allowing you to scale your ClickHouse cluster based on demand.

Networking: Ensure proper network configurations like private endpoints for secure communication with storage or other services.

Considerations: Kubernetes introduces complexity but offers robust management and orchestration features.

Azure Blob Storage as External Storage:

Usage: Instead of using CIFS for data storage, you can configure ClickHouse to use Azure Blob Storage for data, although this setup is more suited for data backup or cold storage rather than primary data storage due to performance differences.

Configuration: ClickHouse has built-in support for Azure Blob Storage with some limitations. You might need to use it alongside local storage for optimal performance.

Custom Docker Images:

Customization: You could create a custom Docker image where you manage permissions or use a different setup to handle file operations, though this won't change the file system used by Azure File Share.

When choosing your approach, consider factors like performance requirements, data consistency, scalability, and management overhead. For file system compatibility, using Azure VMs or AKS where you can directly use ext4 might be the best approach.

Remember, each solution comes with its trade-offs in terms of complexity, cost, and control over the environment.
https://clickhouse.com/blog/clickhouse-cloud-launched-microsoft-launchedsoft-launched
Sid 65 Reputation points

2025-02-02T23:07:17.4666667+00:00

I am using azure container apps. I am yet to try the NFS 4.1. is it compatible with POSIX? I am trying to get a premium storage.
Sid 65 Reputation points

2025-02-02T23:09:39.0833333+00:00

I don’t think permissions is an issue as I have tried giving it the root privileges. In my opinion the file system is the real issue
Prrudram-MSFT 28,201 Reputation points Moderator

2025-02-03T16:52:50.3466667+00:00

@Sid Let me route this to azure container apps service tag for the help. I am not container apps expert.
Sid 65 Reputation points

2025-02-03T16:54:10.86+00:00

Thanks @Prrudram-MSFT
Sid 65 Reputation points

2025-02-03T18:05:51.8266667+00:00

@Prrudram-MSFT quick update - The NFS File share solved the issue, i guess because its POSIX compatible!

1 answer

Your answer

Sid 65 Reputation points

2025-02-02T23:07:17.4666667+00:00

I am using azure container apps. I am yet to try the NFS 4.1. is it compatible with POSIX? I am trying to get a premium storage.
Sid 65 Reputation points

2025-02-02T23:09:39.0833333+00:00

I don’t think permissions is an issue as I have tried giving it the root privileges. In my opinion the file system is the real issue
Prrudram-MSFT 28,201 Reputation points Moderator

2025-02-03T16:52:50.3466667+00:00

@Sid Let me route this to azure container apps service tag for the help. I am not container apps expert.
Sid 65 Reputation points

2025-02-03T16:54:10.86+00:00

Thanks @Prrudram-MSFT
Sid 65 Reputation points

2025-02-03T18:05:51.8266667+00:00

@Prrudram-MSFT quick update - The NFS File share solved the issue, i guess because its POSIX compatible!

Answer 1

Hello @Sid Thanks for the update. I am glad to know it solved your issue. Yes, NFS (Network File System) file shares are POSIX-compatible. For example, Azure Files offers NFS file shares that are fully POSIX-compliant, supporting hard links, symbolic links, and hierarchical storage. However, some features like Kerberos authentication and ACLs (Access Control Lists) might not be supported.
I am posting this as solution, please accept answer and don't forget to thumbs up for "Was it helpful" to benefit others looking for a solution to similar problem.

Good day!

Share via

Issues Inserting Data into ClickHouse on Azure Container

1 answer

Your answer