Monitorar Clusters de Big Data com azdata e kubectl
Este artigo explica como visualizar o status de um cluster de Big Data usando azdata e kubectl.
Importante
O complemento Clusters de Big Data do Microsoft SQL Server 2019 será desativado. O suporte para Clusters de Big Data do SQL Server 2019 será encerrado em 28 de fevereiro de 2025. Todos os usuários existentes do SQL Server 2019 com Software Assurance terão suporte total na plataforma e o software continuará a ser mantido por meio de atualizações cumulativas do SQL Server até esse momento. Para obter mais informações, confira a postagem no blog de anúncio e as opções de Big Data na plataforma do Microsoft SQL Server.
Usar azdata
Você também pode usar comandos azdata para exibir os pontos de extremidade e o status do cluster.
Pontos de extremidade de serviço
Autentique o Cluster de Big Data com azdata login. Defina o parâmetro
--controller-endpointcomo o endereço IP externo do ponto de extremidade do controlador.azdata login --endpoint https://<ip-address-of-controller-svc-external>:30080 --username <user-name>Especifique o nome de usuário e a senha que você configurou para o controlador (AZDATA_USERNAME e AZDATA_PASSWORD) durante a implantação.
Para a autenticação do AD, o comando é:
azdata login --endpoint https://<control_domain_name>:30080 --auth adExecute
azdata bdc endpoint listpara obter uma lista com uma descrição de cada ponto de extremidade e seus valores de porta e endereço IP correspondentes.azdata bdc endpoint list -o tableA lista a seguir mostra a saída de exemplo deste comando:
Description Endpoint Ip Name Port Protocol ------------------------------------------------------ --------------------------------------------------------- -------------- ----------------- ------ ---------- Gateway to access HDFS files, Spark https://11.111.111.111:30443 11.111.111.111 gateway 30443 https Spark Jobs Management and Monitoring Dashboard https://11.111.111.111:30443/gateway/default/sparkhistory 11.111.111.111 spark-history 30443 https Spark Diagnostics and Monitoring Dashboard https://11.111.111.111:30443/gateway/default/yarn 11.111.111.111 yarn-ui 30443 https Application Proxy https://11.111.111.111:30778 11.111.111.111 app-proxy 30778 https Management Proxy https://11.111.111.111:30777 11.111.111.111 mgmtproxy 30777 https Log Search Dashboard https://11.111.111.111:30777/kibana 11.111.111.111 logsui 30777 https Metrics Dashboard https://11.111.111.111:30777/grafana 11.111.111.111 metricsui 30777 https Cluster Management Service https://11.111.111.111:30080 11.111.111.111 controller 30080 https SQL Server Master Instance Front-End 11.111.111.111,31433 11.111.111.111 sql-server-master 31433 tcp HDFS File System Proxy https://11.111.111.111:30443/gateway/default/webhdfs/v1 11.111.111.111 webhdfs 30443 https Proxy for running Spark statements, jobs, applications https://11.111.111.111:30443/gateway/default/livy/v1 11.111.111.111 livy 30443 https
Exibir status do cluster
Você pode exibir o status do cluster com o comando azdata bdc status show.
azdata bdc status show
Dica
Para executar os comandos de status, primeiro você precisa fazer logon usando o comando azdata login, que foi mostrado na seção anterior sobre pontos de extremidade.
Veja a seguir a saída de exemplo deste comando:
Bdc: ready Health Status: healthy
===========================================================================================================================================================================================================================================
Services: ready Health Status: healthy
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Servicename State Healthstatus Details
spark ready healthy -
sql ready healthy -
hdfs ready healthy -
control ready healthy -
gateway ready healthy -
app ready healthy -
Spark Services: ready Health Status: healthy
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Resourcename State Healthstatus Details
sparkhead ready healthy StatefulSet sparkhead is healthy
storage-0 ready healthy StatefulSet storage-0 is healthy
Sql Services: ready Health Status: healthy
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Resourcename State Healthstatus Details
master ready healthy StatefulSet master is healthy
compute-0 ready healthy StatefulSet compute-0 is healthy
data-0 ready healthy StatefulSet data-0 is healthy
storage-0 ready healthy StatefulSet storage-0 is healthy
Hdfs Services: ready Health Status: healthy
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Resourcename State Healthstatus Details
nmnode-0 ready healthy StatefulSet nmnode-0 is healthy
zookeeper ready healthy StatefulSet zookeeper is healthy
storage-0 ready healthy StatefulSet storage-0 is healthy
sparkhead ready healthy StatefulSet sparkhead is healthy
Control Services: ready Health Status: healthy
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Resourcename State Healthstatus Details
controldb ready healthy StatefulSet controldb is healthy
control ready healthy ReplicaSet control is healthy
metricsdc ready healthy DaemonSet metricsdc is healthy
metricsui ready healthy ReplicaSet metricsui is healthy
metricsdb ready healthy StatefulSet metricsdb is healthy
logsui ready healthy ReplicaSet logsui is healthy
logsdb ready healthy StatefulSet logsdb is healthy
mgmtproxy ready healthy ReplicaSet mgmtproxy is healthy
controlwd ready healthy ReplicaSet controlwd is healthy
Gateway Services: ready Health Status: healthy
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Resourcename State Healthstatus Details
gateway ready healthy StatefulSet gateway is healthy
App Services: ready Health Status: healthy
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Resourcename State Healthstatus Details
appproxy ready healthy ReplicaSet appproxy is healthy
Status do recurso específico
Você pode exibir o status de um recurso específico dentro do cluster usando o comando o azdata bdc status show. Ao usar esse comando, você pode filtrar usando o parâmetro --resource. Alguns exemplos de entradas para o parâmetro --resource são:
- master
- controle
- compute-0
- storage-0
- gateway
Por exemplo, o comando a seguir exibe o status do pool de armazenamento:
azdata bdc status show --all --resource storage-0
Para ver o status de todos os componentes que estão executando um serviço específico, use o grupo de comandos correspondente azdata bdc <serviceName> status show. Por exemplo:
azdata bdc sql status show --allazdata bdc hdfs status show --allazdata bdc spark status show --all
Saída de exemplo:
Storage-0: ready Health Status: healthy
===========================================================================================================================================================================================================================================
Instances: running Health Status: healthy
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Instancename State Healthstatus Details
storage-0-0 running healthy Pod storage-0-0 is healthy
storage-0-1 running healthy Pod storage-0-1 is healthy
Dashboards
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Name Url
nodeMetricsUrl https://13.91.50.9:30777/api/v1/bdc/instances/storage-0-1/status/nodemetrics/ui
sqlMetricsUrl https://13.91.50.9:30777/api/v1/bdc/instances/storage-0-1/status/sqlmetrics/ui
logsUrl https://13.91.50.9:30777/api/v1/bdc/instances/storage-0-1/status/logs/ui
Dica
Execute o comando de status com os parâmetros --all para obter detalhes adicionais de integridade, incluindo links para métricas e painéis de logs correspondentes à instância específica. Aqui está um exemplo de saída quando os parâmetros de --all são usados:
Spark: ready Health Status: healthy
===========================================================================================================================================================================================================================================
Resources: ready Health Status: healthy
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Resourcename State Healthstatus Details
sparkhead ready healthy StatefulSet sparkhead is healthy
storage-0 ready healthy StatefulSet storage-0 is healthy
Sparkhead Resources: running Health Status: healthy
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Instancename State Healthstatus Details
sparkhead-0 running healthy Pod sparkhead-0 is healthy
sparkhead-1 running healthy Pod sparkhead-1 is healthy
Dashboards
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Name Url
nodeMetricsUrl https://13.91.50.9:30777/api/v1/bdc/instances/sparkhead-1/status/nodemetrics/ui
sqlMetricsUrl https://13.91.50.9:30777/api/v1/bdc/instances/sparkhead-1/status/sqlmetrics/ui
logsUrl https://13.91.50.9:30777/api/v1/bdc/instances/sparkhead-1/status/logs/ui
Storage-0 Resources: running Health Status: healthy
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Instancename State Healthstatus Details
storage-0-0 running healthy Pod storage-0-0 is healthy
storage-0-1 running healthy Pod storage-0-1 is healthy
Dashboards
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Name Url
nodeMetricsUrl https://13.91.50.9:30777/api/v1/bdc/instances/storage-0-1/status/nodemetrics/ui
sqlMetricsUrl https://13.91.50.9:30777/api/v1/bdc/instances/storage-0-1/status/sqlmetrics/ui
logsUrl https://13.91.50.9:30777/api/v1/bdc/instances/storage-0-1/status/logs/ui
Exibir status do controlador
Você pode exibir o status do controlador com o comando azdata bdc control status show. Ele fornece links semelhantes aos painéis de monitoramento relacionados aos componentes do controlador do cluster de Big Data.