Each Nexus Kubernetes cluster consists of multiple layers:
Virtual Machines (VMs)
Kubernetes layer
Application pods
Figure: Sample Nexus Kubernetes cluster
On an instance, Nexus Kubernetes clusters are delivered with an optionalContainer Insights observability solution.
Container Insights captures the logs and metrics from Nexus Kubernetes clusters and workloads.
It's solely your discretion whether to enable this tooling or deploy your own telemetry stack.
The Nexus Kubernetes cluster with Azure monitoring tool looks like:
Figure: Nexus Kubernetes cluster with Monitoring Tools
Extension onboarding with CLI using managed identity auth
This how-to guide provides steps to enable monitoring agents for the collection of System logs from these VMs using Azure Monitoring Agent.
The instructions further capture details on how to set up log data collection into a Log Analytics workspace.
Prerequisites
Cluster administrator access to the Nexus Kubernetes cluster.
To use Azure Arc-enabled servers, register the following Azure resource providers in your subscription:
Microsoft.HybridCompute
Microsoft.GuestConfiguration
Microsoft.HybridConnectivity
Register these resource providers, if not done previously:
az account set --subscription "{the Subscription Name}"
az provider register --namespace 'Microsoft.HybridCompute'
az provider register --namespace 'Microsoft.GuestConfiguration'
az provider register --namespace 'Microsoft.HybridConnectivity'
Assign an Azure service principal to the following Azure built-in roles, as needed.
Assign the service principal to the Azure resource group that has the machines to be connected:
Needed to deploy the K8s extension for Container Insights
Monitor Nexus Kubernetes cluster
Prerequisites
There are certain prerequisites the operator should ensure to configure the monitoring tools on Nexus Kubernetes Clusters.
Container Insights stores its data in a Log Analytics workspace.
Log data flows into the workspace whose Resource ID you provided during the installation of the Container Insights extension.
Else, data funnels into a default workspace in the Resource group associated with your subscription (based on Azure location).
Run the following command to get a pre-existing Log Analytics workspace Resource ID:
az login
az account set --subscription "<Subscription Name or ID the Log Analytics workspace is in>"
az monitor log-analytics workspace show --workspace-name "<Log Analytics workspace Name>" \
--resource-group "<Log Analytics workspace Resource Group>" \
-o tsv --query id
To deploy
Container Insights and view data in the applicable Log Analytics workspace requires certain role assignments in your account.
For example, the "Contributor" role assignment.
See the instructions for assigning required roles:
Log Analytics Contributor role: necessary permissions to enable container monitoring on a CNF (provisioned) cluster.
Log Analytics Reader role: non-members of the Log Analytics Contributor role, receive permissions to view data in the Log Analytics workspace once you enable container monitoring.
Look for a Provisioning State of "Succeeded" for the extension. The "k8s-extension create" command may have also returned the status.
Customize logs & metrics collection
Container Insights provides end-users functionality to fine-tune the collection of logs and metrics from Nexus Kubernetes Clusters. See the instructions for Configure Container insights agent data collection for more information.
Note
Container Insights does not collect logs from the kube-system namespace by default. To collect logs from the kube-system namespace, you must configure the agent to collect logs from the kube-system namespace.
This can be done by removing the kube-system namespace from the excludedNamespaces field in the ConfigMap following the configMap configuration approach.
[log_collection_settings]
[log_collection_settings.stdout]
# In the absence of this configmap, default value for enabled is true
enabled = true
# exclude_namespaces setting holds good only if enabled is set to true
# kube-system,gatekeeper-system log collection are disabled by default in the absence of 'log_collection_settings.stdout' setting. If you want to enable kube-system,gatekeeper-system, remove them from the following setting.
# If you want to continue to disable kube-system,gatekeeper-system log collection keep the namespaces in the following setting and add any other namespace you want to disable log collection to the array.
# In the absence of this configmap, default value for exclude_namespaces = ["kube-system","gatekeeper-system"]
exclude_namespaces = ["gatekeeper-system"]
# If you want to collect logs from only selective pods inside system namespaces add them to the following setting. Provide namespace:controllerName of the system pod. NOTE: this setting is only for pods in system namespaces
# Valid values for system namespaces are: kube-system, azure-arc, gatekeeper-system, kube-public, kube-node-lease, calico-system. The system namespace used should not be present in exclude_namespaces
# collect_system_pod_logs = ["kube-system:coredns"]
[log_collection_settings.stderr]
# Default value for enabled is true
enabled = true
# exclude_namespaces setting holds good only if enabled is set to true
# kube-system,gatekeeper-system log collection are disabled by default in the absence of 'log_collection_settings.stderr' setting. If you want to enable kube-system,gatekeeper-system, remove them from the following setting.
# If you want to continue to disable kube-system,gatekeeper-system log collection keep the namespaces in the following setting and add any other namespace you want to disable log collection to the array.
# In the absence of this configmap, default value for exclude_namespaces = ["kube-system","gatekeeper-system"]
exclude_namespaces = ["gatekeeper-system"]
# If you want to collect logs from only selective pods inside system namespaces add them to the following setting. Provide namespace:controllerName of the system pod. NOTE: this setting is only for pods in system namespaces
# Valid values for system namespaces are: kube-system, azure-arc, gatekeeper-system, kube-public, kube-node-lease, calico-system. The system namespace used should not be present in exclude_namespaces
# collect_system_pod_logs = ["kube-system:coredns"]
Extra resources
Review workbooks documentation and then you may use Operator Nexus telemetry sample Operator Nexus workbooks.