Prepare Linux for Cache Volumes using a single-node or 2-node cluster (preview)

This article describes how to prepare Linux using a single-node or 2-node cluster, and assumes you fulfilled the prerequisites.

Prepare Linux with AKS enabled by Azure Arc

This section describes how to prepare Linux with AKS enabled by Azure Arc if you run a single-node or 2-node cluster.

1. Install Open Service Mesh (OSM) using the following commands:

   az k8s-extension create --resource-group "YOUR_RESOURCE_GROUP_NAME" --cluster-name "YOUR_CLUSTER_NAME" --cluster-type connectedClusters --extension-type Microsoft.openservicemesh --scope cluster --name osm \
   --config osm.osm.featureFlags.enableWASMStats=false" \
   --config osm.osm.enablePermissiveTrafficPolicy=false" \
   --config osm.osm.configResyncInterval=10s" \
   --config osm.osm.osmController.resource.requests.cpu=100m" \
   --config osm.osm.osmBootstrap.resource.requests.cpu=100m" \
   --config osm.osm.injector.resource.requests.cpu=100m
   

2. Disable ACStor by creating a file named config.json with the following contents:

   {
     "feature.diskStorageClass": "default",
     "acstorController.enabled": false
   }
   

Prepare Linux with AKS Edge Essentials

This section describes how to prepare Linux with AKS Edge Essentials if you run a single-node or two-node cluster.

1. For Edge Essentials to support Azure IoT Operations and Azure Container Storage enabled by Azure Arc, the Kubernetes hosts must be modified to support more memory. You can also increase vCPU and disk allocations at this time if you anticipate requiring additional resources for your Kubernetes uses.

   Start by following the How-To guide here. The QuickStart uses the default configuration and should be avoided.

   Following Step 1: single machine configuration parameters, you have a file in your working directory called aksedge-config.json. Open this file in Notepad or another text editor:

   "SchemaVersion": "1.11",
   "Version": "1.0",
   "DeploymentType": "SingleMachineCluster",
   "Init": {
       "ServiceIPRangeSize": 0
   },
   "Machines": [
   {
       "LinuxNode": {
           "CpuCount": 4,
           "MemoryInMB": 4096,
           "DataSizeInGB": 10,
       }
   }
   ]
   

   Increase MemoryInMB to at least 16384 and DataSizeInGB to 40G. Set ServiceIPRangeSize to 15. If you intend to run many PODs, you can increase the CpuCount as well. For example:

   "Init": {
       "ServiceIPRangeSize": 15
      },
   "Machines": [
   {
       "LinuxNode": {
           "CpuCount": 4,
           "MemoryInMB": 16384,
           "DataSizeInGB": 40,
       }
   }
   ]
   

   Continue with the remaining steps starting with create a single machine cluster. Next, connect your AKS Edge Essentials cluster to Arc.

2. Check for and install Local Path Provisioner storage if it's not already installed. Check if the local-path storage class is already available on your node by running the following cmdlet:

   kubectl get StorageClass
   

   If the local-path storage class is not available, run the following command:

   kubectl apply -f https://raw.githubusercontent.com/Azure/AKS-Edge/main/samples/storage/local-path-provisioner/local-path-storage.yaml
   

   Note

   Local-Path-Provisioner and Busybox images are not maintained by Microsoft and are pulled from the Rancher Labs repository. Local-Path-Provisioner and BusyBox are only available as a Linux container image.

   If everything is correctly configured, you should see the following output:

   NAME                   PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
   local-path (default)   rancher.io/local-path   Delete          WaitForFirstConsumer   false                  21h
   

   If you have multiple disks and want to redirect the path, use:

   kubectl edit configmap -n kube-system local-path-config
   

3. Run the following command to determine if you set fs.inotify.max_user_instances to 1024:

   Invoke-AksEdgeNodeCommand -NodeType "Linux" -Command "sysctl fs.inotify.max_user_instances
   

   After you run this command, if it outputs less than 1024, run the following command to increase the maximum number of files:

   Invoke-AksEdgeNodeCommand -NodeType "Linux" -Command "echo 'fs.inotify.max_user_instances = 1024' | sudo tee -a /etc/sysctl.conf && sudo sysctl -p"
   

4. Install Open Service Mesh (OSM) using the following command:

   az k8s-extension create --resource-group "YOUR_RESOURCE_GROUP_NAME" --cluster-name "YOUR_CLUSTER_NAME" --cluster-type connectedClusters --extension-type Microsoft.openservicemesh --scope cluster --name osm \
   --config "osm.osm.featureFlags.enableWASMStats=false" \
   --config "osm.osm.enablePermissiveTrafficPolicy=false" \
   --config "osm.osm.configResyncInterval=10s" \
   --config "osm.osm.osmController.resource.requests.cpu=100m" \
   --config "osm.osm.osmBootstrap.resource.requests.cpu=100m" \
   --config "osm.osm.injector.resource.requests.cpu=100m
   

5. Disable ACStor by creating a file named config.json with the following contents:

   {
     "acstorController.enabled": false,
     "feature.diskStorageClass": "local-path"
   }
   

Prepare Linux with Ubuntu

This section describes how to prepare Linux with Ubuntu if you run a single-node or two-node cluster.

1. Install Open Service Mesh (OSM) using the following command:

   az k8s-extension create --resource-group "YOUR_RESOURCE_GROUP_NAME" --cluster-name "YOUR_CLUSTER_NAME" --cluster-type connectedClusters --extension-type Microsoft.openservicemesh --scope cluster --name osm \
   --config "osm.osm.featureFlags.enableWASMStats=false" \
   --config "osm.osm.enablePermissiveTrafficPolicy=false" \
   --config "osm.osm.configResyncInterval=10s" \
   --config "osm.osm.osmController.resource.requests.cpu=100m" \
   --config "osm.osm.osmBootstrap.resource.requests.cpu=100m" \
   --config "osm.osm.injector.resource.requests.cpu=100m
   

2. Run the following command to determine if you set fs.inotify.max_user_instances to 1024:

   sysctl fs.inotify.max_user_instances
   

   After you run this command, if it outputs less than 1024, run the following command to increase the maximum number of files and reload the sysctl settings:

   echo 'fs.inotify.max_user_instances = 1024' | sudo tee -a /etc/sysctl.conf
   sudo sysctl -p
   

3. Disable ACStor by creating a file named config.json with the following contents:

   {
     "acstorController.enabled": false,
     "feature.diskStorageClass": "local-path"
   }
   

Next steps

Install Azure Container Storage enabled by Azure Arc