Logs de consulta do Container insights
O Container insights coleta métricas de desempenho, dados de inventário e informações de estado de integridade de hosts e contêineres de contêineres. Os dados são coletados a cada três minutos e encaminhados para o espaço de trabalho do Log Analytics no Azure Monitor, onde estão disponíveis para consultas de log usando o Log Analytics no Azure Monitor.
Você pode aplicar esses dados a cenários que incluem planejamento de migração, análise de capacidade, deteção e solução de problemas de desempenho sob demanda. Os Logs do Monitor do Azure podem ajudá-lo a procurar tendências, diagnosticar gargalos, prever ou correlacionar dados que podem ajudá-lo a determinar se a configuração atual do cluster está funcionando de forma ideal.
Para obter informações sobre como usar essas consultas, consulte Usando consultas no Azure Monitor Log Analytics. Para obter um tutorial completo sobre como usar o Log Analytics para executar consultas e trabalhar com seus resultados, consulte o tutorial do Log Analytics.
Importante
As consultas neste artigo dependem dos dados coletados pelo Container insights e armazenados em um espaço de trabalho do Log Analytics. Se você tiver modificado as configurações padrão de coleta de dados, as consultas podem não retornar os resultados esperados. Mais notavelmente, se você tiver desabilitado a coleta de dados de desempenho desde que ativou as métricas do Prometheus para o cluster, todas as consultas usando a Perf tabela não retornarão resultados.
Consulte Configurar a coleta de dados em Insights de contêiner usando a regra de coleta de dados para configurações predefinidas, incluindo a desativação da coleta de dados de desempenho. Consulte Configurar a coleta de dados no Container insights usando o ConfigMap para obter mais opções de coleta de dados.
Abrir o Log Analytics
Há várias opções para iniciar o Log Analytics. Cada opção começa com um escopo diferente. Para acessar todos os dados no espaço de trabalho, no menu Monitoramento , selecione Logs. Para limitar os dados a um único cluster do Kubernetes, selecione Logs no menu desse cluster.
Consultas dos registos existentes
Você não precisa necessariamente entender como escrever uma consulta de log para usar o Log Analytics. Você pode selecionar entre várias consultas pré-criadas. Você pode executar as consultas sem modificação ou usá-las como um início para uma consulta personalizada. Selecione Consultas na parte superior da tela do Log Analytics e visualize consultas com um tipo de recurso dos Serviços Kubernetes.
Mesas de contentores
Para obter uma lista de tabelas e suas descrições detalhadas usadas pelo Container insights, consulte a referência da tabela do Azure Monitor. Todas essas tabelas estão disponíveis para consultas de log.
Exemplo de consultas dos registos
Muitas vezes, é útil criar consultas que começam com um ou dois exemplos e, em seguida, modificá-las para atender às suas necessidades. Para ajudar a criar consultas mais avançadas, você pode experimentar as seguintes consultas de exemplo.
Listar todas as informações do ciclo de vida de um contêiner
ContainerInventory
| project Computer, Name, Image, ImageTag, ContainerState, CreatedTime, StartedTime, FinishedTime
| render table
Eventos do Kubernetes
Nota
Por padrão, os tipos de evento Normal não são coletados, portanto, você não os verá quando consultar a tabela KubeEvents, a menos que a configuração ConfigMap collect_all_kube_events esteja habilitada. Se você precisar coletar eventos Normal, habilite collect_all_kube_events configuração no container-azm-ms-agentconfig ConfigMap. Consulte Configurar a coleta de dados do agente para insights de contêiner para obter informações sobre como configurar o ConfigMap.
KubeEvents
| where not(isempty(Namespace))
| sort by TimeGenerated desc
| render table
CPU do contêiner
Perf
| where ObjectName == "K8SContainer" and CounterName == "cpuUsageNanoCores"
| summarize AvgCPUUsageNanoCores = avg(CounterValue) by bin(TimeGenerated, 30m), InstanceName
Memória do contentor
Esta consulta usa memoryRssBytes o que só está disponível para nós Linux.
Perf
| where ObjectName == "K8SContainer" and CounterName == "memoryRssBytes"
| summarize AvgUsedRssMemoryBytes = avg(CounterValue) by bin(TimeGenerated, 30m), InstanceName
Solicitações por minuto com métricas personalizadas
InsightsMetrics
| where Name == "requests_count"
| summarize Val=any(Val) by TimeGenerated=bin(TimeGenerated, 1m)
| sort by TimeGenerated asc
| project RequestsPerMinute = Val - prev(Val), TimeGenerated
| render barchart
Pods por nome e namespace
let startTimestamp = ago(1h);
KubePodInventory
| where TimeGenerated > startTimestamp
| project ContainerID, PodName=Name, Namespace
| where PodName contains "name" and Namespace startswith "namespace"
| distinct ContainerID, PodName
| join
(
ContainerLog
| where TimeGenerated > startTimestamp
)
on ContainerID
// at this point before the next pipe, columns from both tables are available to be "projected". Due to both
// tables having a "Name" column, we assign an alias as PodName to one column which we actually want
| project TimeGenerated, PodName, LogEntry, LogEntrySource
| summarize by TimeGenerated, LogEntry
| order by TimeGenerated desc
Expansão do pod (HPA)
Essa consulta retorna o número de réplicas dimensionadas em cada implantação. Ele calcula a porcentagem de expansão com o número máximo de réplicas configuradas no HPA.
let _minthreshold = 70; // minimum threshold goes here if you want to setup as an alert
let _maxthreshold = 90; // maximum threshold goes here if you want to setup as an alert
let startDateTime = ago(60m);
KubePodInventory
| where TimeGenerated >= startDateTime
| where Namespace !in('default', 'kube-system') // List of non system namespace filter goes here.
| extend labels = todynamic(PodLabel)
| extend deployment_hpa = reverse(substring(reverse(ControllerName), indexof(reverse(ControllerName), "-") + 1))
| distinct tostring(deployment_hpa)
| join kind=inner (InsightsMetrics
| where TimeGenerated > startDateTime
| where Name == 'kube_hpa_status_current_replicas'
| extend pTags = todynamic(Tags) //parse the tags for values
| extend ns = todynamic(pTags.k8sNamespace) //parse namespace value from tags
| extend deployment_hpa = todynamic(pTags.targetName) //parse HPA target name from tags
| extend max_reps = todynamic(pTags.spec_max_replicas) // Parse maximum replica settings from HPA deployment
| extend desired_reps = todynamic(pTags.status_desired_replicas) // Parse desired replica settings from HPA deployment
| summarize arg_max(TimeGenerated, *) by tostring(ns), tostring(deployment_hpa), Cluster=toupper(tostring(split(_ResourceId, '/')[8])), toint(desired_reps), toint(max_reps), scale_out_percentage=(desired_reps * 100 / max_reps)
//| where scale_out_percentage > _minthreshold and scale_out_percentage <= _maxthreshold
)
on deployment_hpa
Scale-outs do Nodepool
Essa consulta retorna o número de nós ativos em cada pool de nós. Ele calcula o número de nós ativos disponíveis e a configuração máxima do nó nas configurações do autoscaler para determinar a porcentagem de expansão. Consulte as linhas comentadas na consulta para usá-la para uma regra de alerta de vários resultados .
let nodepoolMaxnodeCount = 10; // the maximum number of nodes in your auto scale setting goes here.
let _minthreshold = 20;
let _maxthreshold = 90;
let startDateTime = 60m;
KubeNodeInventory
| where TimeGenerated >= ago(startDateTime)
| extend nodepoolType = todynamic(Labels) //Parse the labels to get the list of node pool types
| extend nodepoolName = todynamic(nodepoolType[0].agentpool) // parse the label to get the nodepool name or set the specific nodepool name (like nodepoolName = 'agentpool)'
| summarize nodeCount = count(Computer) by ClusterName, tostring(nodepoolName), TimeGenerated
//(Uncomment the below two lines to set this as a log search alert)
//| extend scaledpercent = iff(((nodeCount * 100 / nodepoolMaxnodeCount) >= _minthreshold and (nodeCount * 100 / nodepoolMaxnodeCount) < _maxthreshold), "warn", "normal")
//| where scaledpercent == 'warn'
| summarize arg_max(TimeGenerated, *) by nodeCount, ClusterName, tostring(nodepoolName)
| project ClusterName,
TotalNodeCount= strcat("Total Node Count: ", nodeCount),
ScaledOutPercentage = (nodeCount * 100 / nodepoolMaxnodeCount),
TimeGenerated,
nodepoolName
Disponibilidade de contêineres do sistema (conjunto de réplicas)
Essa consulta retorna os contêineres do sistema (replicasets) e relata a porcentagem indisponível. Consulte as linhas comentadas na consulta para usá-la para uma regra de alerta de vários resultados .
let startDateTime = 5m; // the minimum time interval goes here
let _minalertThreshold = 50; //Threshold for minimum and maximum unavailable or not running containers
let _maxalertThreshold = 70;
KubePodInventory
| where TimeGenerated >= ago(startDateTime)
| distinct ClusterName, TimeGenerated
| summarize Clustersnapshot = count() by ClusterName
| join kind=inner (
KubePodInventory
| where TimeGenerated >= ago(startDateTime)
| where Namespace in('default', 'kube-system') and ControllerKind == 'ReplicaSet' // the system namespace filter goes here
| distinct ClusterName, Computer, PodUid, TimeGenerated, PodStatus, ServiceName, PodLabel, Namespace, ContainerStatus
| summarize arg_max(TimeGenerated, *), TotalPODCount = count(), podCount = sumif(1, PodStatus == 'Running' or PodStatus != 'Running'), containerNotrunning = sumif(1, ContainerStatus != 'running')
by ClusterName, TimeGenerated, ServiceName, PodLabel, Namespace
)
on ClusterName
| project ClusterName, ServiceName, podCount, containerNotrunning, containerNotrunningPercent = (containerNotrunning * 100 / podCount), TimeGenerated, PodStatus, PodLabel, Namespace, Environment = tostring(split(ClusterName, '-')[3]), Location = tostring(split(ClusterName, '-')[4]), ContainerStatus
//Uncomment the below line to set for automated alert
//| where PodStatus == "Running" and containerNotrunningPercent > _minalertThreshold and containerNotrunningPercent < _maxalertThreshold
| summarize arg_max(TimeGenerated, *), c_entry=count() by PodLabel, ServiceName, ClusterName
//Below lines are to parse the labels to identify the impacted service/component name
| extend parseLabel = replace(@'k8s-app', @'k8sapp', PodLabel)
| extend parseLabel = replace(@'app.kubernetes.io\\/component', @'appkubernetesiocomponent', parseLabel)
| extend parseLabel = replace(@'app.kubernetes.io\\/instance', @'appkubernetesioinstance', parseLabel)
| extend tags = todynamic(parseLabel)
| extend tag01 = todynamic(tags[0].app)
| extend tag02 = todynamic(tags[0].k8sapp)
| extend tag03 = todynamic(tags[0].appkubernetesiocomponent)
| extend tag04 = todynamic(tags[0].aadpodidbinding)
| extend tag05 = todynamic(tags[0].appkubernetesioinstance)
| extend tag06 = todynamic(tags[0].component)
| project ClusterName, TimeGenerated,
ServiceName = strcat( ServiceName, tag01, tag02, tag03, tag04, tag05, tag06),
ContainerUnavailable = strcat("Unavailable Percentage: ", containerNotrunningPercent),
PodStatus = strcat("PodStatus: ", PodStatus),
ContainerStatus = strcat("Container Status: ", ContainerStatus)
Disponibilidade de contêineres do sistema (daemonsets)
Essa consulta retorna os contêineres do sistema (daemonsets) e relata a porcentagem indisponível. Consulte as linhas comentadas na consulta para usá-la para uma regra de alerta de vários resultados .
let startDateTime = 5m; // the minimum time interval goes here
let _minalertThreshold = 50; //Threshold for minimum and maximum unavailable or not running containers
let _maxalertThreshold = 70;
KubePodInventory
| where TimeGenerated >= ago(startDateTime)
| distinct ClusterName, TimeGenerated
| summarize Clustersnapshot = count() by ClusterName
| join kind=inner (
KubePodInventory
| where TimeGenerated >= ago(startDateTime)
| where Namespace in('default', 'kube-system') and ControllerKind == 'DaemonSet' // the system namespace filter goes here
| distinct ClusterName, Computer, PodUid, TimeGenerated, PodStatus, ServiceName, PodLabel, Namespace, ContainerStatus
| summarize arg_max(TimeGenerated, *), TotalPODCount = count(), podCount = sumif(1, PodStatus == 'Running' or PodStatus != 'Running'), containerNotrunning = sumif(1, ContainerStatus != 'running')
by ClusterName, TimeGenerated, ServiceName, PodLabel, Namespace
)
on ClusterName
| project ClusterName, ServiceName, podCount, containerNotrunning, containerNotrunningPercent = (containerNotrunning * 100 / podCount), TimeGenerated, PodStatus, PodLabel, Namespace, Environment = tostring(split(ClusterName, '-')[3]), Location = tostring(split(ClusterName, '-')[4]), ContainerStatus
//Uncomment the below line to set for automated alert
//| where PodStatus == "Running" and containerNotrunningPercent > _minalertThreshold and containerNotrunningPercent < _maxalertThreshold
| summarize arg_max(TimeGenerated, *), c_entry=count() by PodLabel, ServiceName, ClusterName
//Below lines are to parse the labels to identify the impacted service/component name
| extend parseLabel = replace(@'k8s-app', @'k8sapp', PodLabel)
| extend parseLabel = replace(@'app.kubernetes.io\\/component', @'appkubernetesiocomponent', parseLabel)
| extend parseLabel = replace(@'app.kubernetes.io\\/instance', @'appkubernetesioinstance', parseLabel)
| extend tags = todynamic(parseLabel)
| extend tag01 = todynamic(tags[0].app)
| extend tag02 = todynamic(tags[0].k8sapp)
| extend tag03 = todynamic(tags[0].appkubernetesiocomponent)
| extend tag04 = todynamic(tags[0].aadpodidbinding)
| extend tag05 = todynamic(tags[0].appkubernetesioinstance)
| extend tag06 = todynamic(tags[0].component)
| project ClusterName, TimeGenerated,
ServiceName = strcat( ServiceName, tag01, tag02, tag03, tag04, tag05, tag06),
ContainerUnavailable = strcat("Unavailable Percentage: ", containerNotrunningPercent),
PodStatus = strcat("PodStatus: ", PodStatus),
ContainerStatus = strcat("Container Status: ", ContainerStatus)
Logs de contêiner
Os logs de contêiner para AKS são armazenados na tabela ContainerLogV2. Você pode executar as seguintes consultas de exemplo para procurar a saída de log stderr/stdout de pods de destino, implantações ou namespaces.
Logs de contêiner para um pod, namespace e contêiner específicos
ContainerLogV2
| where _ResourceId =~ "clusterResourceID" //update with resource ID
| where PodNamespace == "podNameSpace" //update with target namespace
| where PodName == "podName" //update with target pod
| where ContainerName == "containerName" //update with target container
| project TimeGenerated, Computer, ContainerId, LogMessage, LogSource
Logs de contêiner para uma implantação específica
let KubePodInv = KubePodInventory
| where _ResourceId =~ "clusterResourceID" //update with resource ID
| where Namespace == "deploymentNamespace" //update with target namespace
| where ControllerKind == "ReplicaSet"
| extend deployment = reverse(substring(reverse(ControllerName), indexof(reverse(ControllerName), "-") + 1))
| where deployment == "deploymentName" //update with target deployment
| extend ContainerId = ContainerID
| summarize arg_max(TimeGenerated, *) by deployment, ContainerId, PodStatus, ContainerStatus
| project deployment, ContainerId, PodStatus, ContainerStatus;
KubePodInv
| join
(
ContainerLogV2
| where TimeGenerated >= startTime and TimeGenerated < endTime
| where PodNamespace == "deploymentNamespace" //update with target namespace
| where PodName startswith "deploymentName" //update with target deployment
) on ContainerId
| project TimeGenerated, deployment, PodName, PodStatus, ContainerName, ContainerId, ContainerStatus, LogMessage, LogSource
Logs de contêiner para qualquer pod com falha em um namespace específico
let KubePodInv = KubePodInventory
| where TimeGenerated >= startTime and TimeGenerated < endTime
| where _ResourceId =~ "clustereResourceID" //update with resource ID
| where Namespace == "podNamespace" //update with target namespace
| where PodStatus == "Failed"
| extend ContainerId = ContainerID
| summarize arg_max(TimeGenerated, *) by ContainerId, PodStatus, ContainerStatus
| project ContainerId, PodStatus, ContainerStatus;
KubePodInv
| join
(
ContainerLogV2
| where TimeGenerated >= startTime and TimeGenerated < endTime
| where PodNamespace == "podNamespace" //update with target namespace
) on ContainerId
| project TimeGenerated, PodName, PodStatus, ContainerName, ContainerId, ContainerStatus, LogMessage, LogSource
Consultas de visualização padrão de insights de contêiner
Essas consultas são geradas a partir de visualizações prontas para uso a partir de insights de contêiner. Você pode optar por usá-los se tiver ativado configurações personalizadas de otimização de custos, em vez dos gráficos padrão.
Contagem de nós por status
As tabelas necessárias para este gráfico incluem KubeNodeInventory.
let trendBinSize = 5m;
let maxListSize = 1000;
let clusterId = 'clusterResourceID'; //update with resource ID
let rawData = KubeNodeInventory
| where ClusterId =~ clusterId
| distinct ClusterId, TimeGenerated
| summarize ClusterSnapshotCount = count() by Timestamp = bin(TimeGenerated, trendBinSize), ClusterId
| join hint.strategy=broadcast ( KubeNodeInventory
| where ClusterId =~ clusterId
| summarize TotalCount = count(), ReadyCount = sumif(1, Status contains ('Ready')) by ClusterId, Timestamp = bin(TimeGenerated, trendBinSize)
| extend NotReadyCount = TotalCount - ReadyCount ) on ClusterId, Timestamp
| project ClusterId, Timestamp, TotalCount = todouble(TotalCount) / ClusterSnapshotCount, ReadyCount = todouble(ReadyCount) / ClusterSnapshotCount, NotReadyCount = todouble(NotReadyCount) / ClusterSnapshotCount;
rawData
| order by Timestamp asc
| summarize makelist(Timestamp, maxListSize), makelist(TotalCount, maxListSize), makelist(ReadyCount, maxListSize), makelist(NotReadyCount, maxListSize) by ClusterId
| join ( rawData
| summarize Avg_TotalCount = avg(TotalCount), Avg_ReadyCount = avg(ReadyCount), Avg_NotReadyCount = avg(NotReadyCount) by ClusterId ) on ClusterId
| project ClusterId, Avg_TotalCount, Avg_ReadyCount, Avg_NotReadyCount, list_Timestamp, list_TotalCount, list_ReadyCount, list_NotReadyCount
Contagem de pods por status
As tabelas necessárias para este gráfico incluem KubePodInventory.
let trendBinSize = 5m;
let maxListSize = 1000;
let clusterId = 'clusterResourceID'; //update with resource ID
let rawData = KubePodInventory
| where ClusterId =~ clusterId
| distinct ClusterId, TimeGenerated
| summarize ClusterSnapshotCount = count() by bin(TimeGenerated, trendBinSize), ClusterId
| join hint.strategy=broadcast ( KubePodInventory
| where ClusterId =~ clusterId
| summarize PodStatus=any(PodStatus) by TimeGenerated, PodUid, ClusterId
| summarize TotalCount = count(), PendingCount = sumif(1, PodStatus =~ 'Pending'), RunningCount = sumif(1, PodStatus =~ 'Running'), SucceededCount = sumif(1, PodStatus =~ 'Succeeded'), FailedCount = sumif(1, PodStatus =~ 'Failed'), TerminatingCount = sumif(1, PodStatus =~ 'Terminating') by ClusterId, bin(TimeGenerated, trendBinSize) ) on ClusterId, TimeGenerated
| extend UnknownCount = TotalCount - PendingCount - RunningCount - SucceededCount - FailedCount - TerminatingCount
| project ClusterId, Timestamp = TimeGenerated, TotalCount = todouble(TotalCount) / ClusterSnapshotCount, PendingCount = todouble(PendingCount) / ClusterSnapshotCount, RunningCount = todouble(RunningCount) / ClusterSnapshotCount, SucceededCount = todouble(SucceededCount) / ClusterSnapshotCount, FailedCount = todouble(FailedCount) / ClusterSnapshotCount, TerminatingCount = todouble(TerminatingCount) / ClusterSnapshotCount, UnknownCount = todouble(UnknownCount) / ClusterSnapshotCount;
let rawDataCached = rawData;
rawDataCached
| order by Timestamp asc
| summarize makelist(Timestamp, maxListSize), makelist(TotalCount, maxListSize), makelist(PendingCount, maxListSize), makelist(RunningCount, maxListSize), makelist(SucceededCount, maxListSize), makelist(FailedCount, maxListSize), makelist(TerminatingCount, maxListSize), makelist(UnknownCount, maxListSize) by ClusterId
| join ( rawDataCached
| summarize Avg_TotalCount = avg(TotalCount), Avg_PendingCount = avg(PendingCount), Avg_RunningCount = avg(RunningCount), Avg_SucceededCount = avg(SucceededCount), Avg_FailedCount = avg(FailedCount), Avg_TerminatingCount = avg(TerminatingCount), Avg_UnknownCount = avg(UnknownCount) by ClusterId ) on ClusterId
| project ClusterId, Avg_TotalCount, Avg_PendingCount, Avg_RunningCount, Avg_SucceededCount, Avg_FailedCount, Avg_TerminatingCount, Avg_UnknownCount, list_Timestamp, list_TotalCount, list_PendingCount, list_RunningCount, list_SucceededCount, list_FailedCount, list_TerminatingCount, list_UnknownCount
Lista de contentores por estado
As tabelas necessárias para este gráfico incluem KubePodInventory e Perf.
let startDateTime = datetime('start time');
let endDateTime = datetime('end time');
let trendBinSize = 15m;
let maxResultCount = 10000;
let metricUsageCounterName = 'cpuUsageNanoCores';
let metricLimitCounterName = 'cpuLimitNanoCores';
let KubePodInventoryTable = KubePodInventory
| where TimeGenerated >= startDateTime
| where TimeGenerated < endDateTime
| where isnotempty(ClusterName)
| where isnotempty(Namespace)
| where isnotempty(Computer)
| project TimeGenerated, ClusterId, ClusterName, Namespace, ServiceName, ControllerName, Node = Computer, Pod = Name, ContainerInstance = ContainerName, ContainerID, ReadySinceNow = format_timespan(endDateTime - ContainerCreationTimeStamp , 'ddd.hh:mm:ss.fff'), Restarts = ContainerRestartCount, Status = ContainerStatus, ContainerStatusReason = columnifexists('ContainerStatusReason', ''), ControllerKind = ControllerKind, PodStatus;
let startRestart = KubePodInventoryTable
| summarize arg_min(TimeGenerated, *) by Node, ContainerInstance
| where ClusterId =~ 'clusterResourceID' //update with resource ID
| project Node, ContainerInstance, InstanceName = strcat(ClusterId, '/', ContainerInstance), StartRestart = Restarts;
let IdentityTable = KubePodInventoryTable
| summarize arg_max(TimeGenerated, *) by Node, ContainerInstance
| where ClusterId =~ 'clusterResourceID' //update with resource ID
| project ClusterName, Namespace, ServiceName, ControllerName, Node, Pod, ContainerInstance, InstanceName = strcat(ClusterId, '/', ContainerInstance), ContainerID, ReadySinceNow, Restarts, Status = iff(Status =~ 'running', 0, iff(Status=~'waiting', 1, iff(Status =~'terminated', 2, 3))), ContainerStatusReason, ControllerKind, Containers = 1, ContainerName = tostring(split(ContainerInstance, '/')[1]), PodStatus, LastPodInventoryTimeGenerated = TimeGenerated, ClusterId;
let CachedIdentityTable = IdentityTable;
let FilteredPerfTable = Perf
| where TimeGenerated >= startDateTime
| where TimeGenerated < endDateTime
| where ObjectName == 'K8SContainer'
| where InstanceName startswith 'clusterResourceID'
| project Node = Computer, TimeGenerated, CounterName, CounterValue, InstanceName ;
let CachedFilteredPerfTable = FilteredPerfTable;
let LimitsTable = CachedFilteredPerfTable
| where CounterName =~ metricLimitCounterName
| summarize arg_max(TimeGenerated, *) by Node, InstanceName
| project Node, InstanceName, LimitsValue = iff(CounterName =~ 'cpuLimitNanoCores', CounterValue/1000000, CounterValue), TimeGenerated;
let MetaDataTable = CachedIdentityTable
| join kind=leftouter ( LimitsTable ) on Node, InstanceName
| join kind= leftouter ( startRestart ) on Node, InstanceName
| project ClusterName, Namespace, ServiceName, ControllerName, Node, Pod, InstanceName, ContainerID, ReadySinceNow, Restarts, LimitsValue, Status, ContainerStatusReason = columnifexists('ContainerStatusReason', ''), ControllerKind, Containers, ContainerName, ContainerInstance, StartRestart, PodStatus, LastPodInventoryTimeGenerated, ClusterId;
let UsagePerfTable = CachedFilteredPerfTable
| where CounterName =~ metricUsageCounterName
| project TimeGenerated, Node, InstanceName, CounterValue = iff(CounterName =~ 'cpuUsageNanoCores', CounterValue/1000000, CounterValue);
let LastRestartPerfTable = CachedFilteredPerfTable
| where CounterName =~ 'restartTimeEpoch'
| summarize arg_max(TimeGenerated, *) by Node, InstanceName
| project Node, InstanceName, UpTime = CounterValue, LastReported = TimeGenerated;
let AggregationTable = UsagePerfTable
| summarize Aggregation = max(CounterValue) by Node, InstanceName
| project Node, InstanceName, Aggregation;
let TrendTable = UsagePerfTable
| summarize TrendAggregation = max(CounterValue) by bin(TimeGenerated, trendBinSize), Node, InstanceName
| project TrendTimeGenerated = TimeGenerated, Node, InstanceName , TrendAggregation
| summarize TrendList = makelist(pack("timestamp", TrendTimeGenerated, "value", TrendAggregation)) by Node, InstanceName;
let containerFinalTable = MetaDataTable
| join kind= leftouter( AggregationTable ) on Node, InstanceName
| join kind = leftouter (LastRestartPerfTable) on Node, InstanceName
| order by Aggregation desc, ContainerName
| join kind = leftouter ( TrendTable) on Node, InstanceName
| extend ContainerIdentity = strcat(ContainerName, ' ', Pod)
| project ContainerIdentity, Status, ContainerStatusReason = columnifexists('ContainerStatusReason', ''), Aggregation, Node, Restarts, ReadySinceNow, TrendList = iif(isempty(TrendList), parse_json('[]'), TrendList), LimitsValue, ControllerName, ControllerKind, ContainerID, Containers, UpTimeNow = datetime_diff('Millisecond', endDateTime, datetime_add('second', toint(UpTime), make_datetime(1970,1,1))), ContainerInstance, StartRestart, LastReportedDelta = datetime_diff('Millisecond', endDateTime, LastReported), PodStatus, InstanceName, Namespace, LastPodInventoryTimeGenerated, ClusterId;
containerFinalTable
| limit 200
Lista de controladores por status
As tabelas necessárias para este gráfico incluem KubePodInventory e Perf.
let endDateTime = datetime('start time');
let startDateTime = datetime('end time');
let trendBinSize = 15m;
let metricLimitCounterName = 'cpuLimitNanoCores';
let metricUsageCounterName = 'cpuUsageNanoCores';
let primaryInventory = KubePodInventory
| where TimeGenerated >= startDateTime
| where TimeGenerated < endDateTime
| where isnotempty(ClusterName)
| where isnotempty(Namespace)
| extend Node = Computer
| where ClusterId =~ 'clusterResourceID' //update with resource ID
| project TimeGenerated, ClusterId, ClusterName, Namespace, ServiceName, Node = Computer, ControllerName, Pod = Name, ContainerInstance = ContainerName, ContainerID, InstanceName, PerfJoinKey = strcat(ClusterId, '/', ContainerName), ReadySinceNow = format_timespan(endDateTime - ContainerCreationTimeStamp, 'ddd.hh:mm:ss.fff'), Restarts = ContainerRestartCount, Status = ContainerStatus, ContainerStatusReason = columnifexists('ContainerStatusReason', ''), ControllerKind = ControllerKind, PodStatus, ControllerId = strcat(ClusterId, '/', Namespace, '/', ControllerName);
let podStatusRollup = primaryInventory
| summarize arg_max(TimeGenerated, *) by Pod
| project ControllerId, PodStatus, TimeGenerated
| summarize count() by ControllerId, PodStatus = iif(TimeGenerated < ago(30m), 'Unknown', PodStatus)
| summarize PodStatusList = makelist(pack('Status', PodStatus, 'Count', count_)) by ControllerId;
let latestContainersByController = primaryInventory
| where isnotempty(Node)
| summarize arg_max(TimeGenerated, *) by PerfJoinKey
| project ControllerId, PerfJoinKey;
let filteredPerformance = Perf
| where TimeGenerated >= startDateTime
| where TimeGenerated < endDateTime
| where ObjectName == 'K8SContainer'
| where InstanceName startswith 'clusterResourceID' //update with resource ID
| project TimeGenerated, CounterName, CounterValue, InstanceName, Node = Computer ;
let metricByController = filteredPerformance
| where CounterName =~ metricUsageCounterName
| extend PerfJoinKey = InstanceName
| summarize Value = percentile(CounterValue, 95) by PerfJoinKey, CounterName
| join (latestContainersByController) on PerfJoinKey
| summarize Value = sum(Value) by ControllerId, CounterName
| project ControllerId, CounterName, AggregationValue = iff(CounterName =~ 'cpuUsageNanoCores', Value/1000000, Value);
let containerCountByController = latestContainersByController
| summarize ContainerCount = count() by ControllerId;
let restartCountsByController = primaryInventory
| summarize Restarts = max(Restarts) by ControllerId;
let oldestRestart = primaryInventory
| summarize ReadySinceNow = min(ReadySinceNow) by ControllerId;
let trendLineByController = filteredPerformance
| where CounterName =~ metricUsageCounterName
| extend PerfJoinKey = InstanceName
| summarize Value = percentile(CounterValue, 95) by bin(TimeGenerated, trendBinSize), PerfJoinKey, CounterName
| order by TimeGenerated asc
| join kind=leftouter (latestContainersByController) on PerfJoinKey
| summarize Value=sum(Value) by ControllerId, TimeGenerated, CounterName
| project TimeGenerated, Value = iff(CounterName =~ 'cpuUsageNanoCores', Value/1000000, Value), ControllerId
| summarize TrendList = makelist(pack("timestamp", TimeGenerated, "value", Value)) by ControllerId;
let latestLimit = filteredPerformance
| where CounterName =~ metricLimitCounterName
| extend PerfJoinKey = InstanceName
| summarize arg_max(TimeGenerated, *) by PerfJoinKey
| join kind=leftouter (latestContainersByController) on PerfJoinKey
| summarize Value = sum(CounterValue) by ControllerId, CounterName
| project ControllerId, LimitValue = iff(CounterName =~ 'cpuLimitNanoCores', Value/1000000, Value);
let latestTimeGeneratedByController = primaryInventory
| summarize arg_max(TimeGenerated, *) by ControllerId
| project ControllerId, LastTimeGenerated = TimeGenerated;
primaryInventory
| distinct ControllerId, ControllerName, ControllerKind, Namespace
| join kind=leftouter (podStatusRollup) on ControllerId
| join kind=leftouter (metricByController) on ControllerId
| join kind=leftouter (containerCountByController) on ControllerId
| join kind=leftouter (restartCountsByController) on ControllerId
| join kind=leftouter (oldestRestart) on ControllerId
| join kind=leftouter (trendLineByController) on ControllerId
| join kind=leftouter (latestLimit) on ControllerId
| join kind=leftouter (latestTimeGeneratedByController) on ControllerId
| project ControllerId, ControllerName, ControllerKind, PodStatusList, AggregationValue, ContainerCount = iif(isempty(ContainerCount), 0, ContainerCount), Restarts, ReadySinceNow, Node = '-', TrendList, LimitValue, LastTimeGenerated, Namespace
| limit 250;
Lista de nós por status
As tabelas necessárias para este gráfico incluem KubeNodeInventory, KubePodInventory e Perf.
let endDateTime = datetime('start time');
let startDateTime = datetime('end time');
let binSize = 15m;
let limitMetricName = 'cpuCapacityNanoCores';
let usedMetricName = 'cpuUsageNanoCores';
let materializedNodeInventory = KubeNodeInventory
| where TimeGenerated < endDateTime
| where TimeGenerated >= startDateTime
| project ClusterName, ClusterId, Node = Computer, TimeGenerated, Status, NodeName = Computer, NodeId = strcat(ClusterId, '/', Computer), Labels
| where ClusterId =~ 'clusterResourceID'; //update with resource ID
let materializedPerf = Perf
| where TimeGenerated < endDateTime
| where TimeGenerated >= startDateTime
| where ObjectName == 'K8SNode'
| extend NodeId = InstanceName;
let materializedPodInventory = KubePodInventory
| where TimeGenerated < endDateTime
| where TimeGenerated >= startDateTime
| where isnotempty(ClusterName)
| where isnotempty(Namespace)
| where ClusterId =~ 'clusterResourceID'; //update with resource ID
let inventoryOfCluster = materializedNodeInventory
| summarize arg_max(TimeGenerated, Status) by ClusterName, ClusterId, NodeName, NodeId;
let labelsByNode = materializedNodeInventory
| summarize arg_max(TimeGenerated, Labels) by ClusterName, ClusterId, NodeName, NodeId;
let countainerCountByNode = materializedPodInventory
| project ContainerName, NodeId = strcat(ClusterId, '/', Computer)
| distinct NodeId, ContainerName
| summarize ContainerCount = count() by NodeId;
let latestUptime = materializedPerf
| where CounterName == 'restartTimeEpoch'
| summarize arg_max(TimeGenerated, CounterValue) by NodeId
| extend UpTimeMs = datetime_diff('Millisecond', endDateTime, datetime_add('second', toint(CounterValue), make_datetime(1970,1,1)))
| project NodeId, UpTimeMs;
let latestLimitOfNodes = materializedPerf
| where CounterName == limitMetricName
| summarize CounterValue = max(CounterValue) by NodeId
| project NodeId, LimitValue = CounterValue;
let actualUsageAggregated = materializedPerf
| where CounterName == usedMetricName
| summarize Aggregation = percentile(CounterValue, 95) by NodeId //This line updates to the desired aggregation
| project NodeId, Aggregation;
let aggregateTrendsOverTime = materializedPerf
| where CounterName == usedMetricName
| summarize TrendAggregation = percentile(CounterValue, 95) by NodeId, bin(TimeGenerated, binSize) //This line updates to the desired aggregation
| project NodeId, TrendAggregation, TrendDateTime = TimeGenerated;
let unscheduledPods = materializedPodInventory
| where isempty(Computer)
| extend Node = Computer
| where isempty(ContainerStatus)
| where PodStatus == 'Pending'
| order by TimeGenerated desc
| take 1
| project ClusterName, NodeName = 'unscheduled', LastReceivedDateTime = TimeGenerated, Status = 'unscheduled', ContainerCount = 0, UpTimeMs = '0', Aggregation = '0', LimitValue = '0', ClusterId;
let scheduledPods = inventoryOfCluster
| join kind=leftouter (aggregateTrendsOverTime) on NodeId
| extend TrendPoint = pack("TrendTime", TrendDateTime, "TrendAggregation", TrendAggregation)
| summarize make_list(TrendPoint) by NodeId, NodeName, Status
| join kind=leftouter (labelsByNode) on NodeId
| join kind=leftouter (countainerCountByNode) on NodeId
| join kind=leftouter (latestUptime) on NodeId
| join kind=leftouter (latestLimitOfNodes) on NodeId
| join kind=leftouter (actualUsageAggregated) on NodeId
| project ClusterName, NodeName, ClusterId, list_TrendPoint, LastReceivedDateTime = TimeGenerated, Status, ContainerCount, UpTimeMs, Aggregation, LimitValue, Labels
| limit 250;
union (scheduledPods), (unscheduledPods)
| project ClusterName, NodeName, LastReceivedDateTime, Status, ContainerCount, UpTimeMs = UpTimeMs_long, Aggregation = Aggregation_real, LimitValue = LimitValue_real, list_TrendPoint, Labels, ClusterId
Métricas Prometheus
Os exemplos a seguir exigem a configuração descrita em Enviar métricas do Prometheus para o espaço de trabalho do Log Analytics com insights de contêiner.
Para exibir métricas do Prometheus raspadas pelo Azure Monitor e filtradas por namespace, especifique "prometheus". Aqui está uma consulta de exemplo para exibir as métricas do Prometheus do default namespace do Kubernetes.
InsightsMetrics
| where Namespace contains "prometheus"
| extend tags=parse_json(Tags)
| summarize count() by Name
Os dados do Prometheus também podem ser consultados diretamente pelo nome.
InsightsMetrics
| where Namespace contains "prometheus"
| where Name contains "some_prometheus_metric"
Para identificar o volume de ingestão de cada tamanho de métrica em GB por dia para entender se ele é alto, a consulta a seguir é fornecida.
InsightsMetrics
| where Namespace contains "prometheus"
| where TimeGenerated > ago(24h)
| summarize VolumeInGB = (sum(_BilledSize) / (1024 * 1024 * 1024)) by Name
| order by VolumeInGB desc
| render barchart
A saída mostrará resultados semelhantes ao exemplo a seguir.
Para estimar qual é o tamanho de cada métrica em GB por um mês Para entender se o volume de dados ingeridos recebidos no espaço de trabalho é alto, a consulta a seguir é fornecida.
InsightsMetrics
| where Namespace contains "prometheus"
| where TimeGenerated > ago(24h)
| summarize EstimatedGBPer30dayMonth = (sum(_BilledSize) / (1024 * 1024 * 1024)) * 30 by Name
| order by EstimatedGBPer30dayMonth desc
| render barchart
A saída mostrará resultados semelhantes ao exemplo a seguir.
Erros de configuração ou raspagem
Para investigar quaisquer erros de configuração ou raspagem, a consulta de exemplo a seguir retorna eventos informativos da KubeMonAgentEvents tabela.
KubeMonAgentEvents | where Level != "Info"
A saída mostra resultados semelhantes ao exemplo a seguir:
Perguntas mais frequentes
Esta secção fornece respostas a perguntas comuns.
Posso visualizar as métricas coletadas no Grafana?
Os insights de contêiner suportam a visualização de métricas armazenadas em seu espaço de trabalho do Log Analytics nos painéis do Grafana. Fornecemos um modelo que você pode baixar do repositório do painel do Grafana. Use-o para começar e como referência para ajudá-lo a aprender a consultar dados de seus clusters monitorados para visualizar em painéis personalizados do Grafana.
Por que as linhas de log maiores que 16 KB são divididas em vários registros no Log Analytics?
O agente usa o driver de log de arquivos JSON do Docker para capturar o stdout e o stderr de contêineres. Esse driver de log divide linhas de log maiores que 16 KB em várias linhas quando elas são copiadas de stdout ou stderr para um arquivo.
Próximos passos
O Container insights não inclui um conjunto predefinido de alertas. Para saber como criar alertas recomendados para alta utilização de CPU e memória para dar suporte ao seu DevOps ou processos e procedimentos operacionais, consulte Criar alertas de desempenho com insights de contêiner.