How can I enable requests from within an AKS cluster to read services in a different cluster privately?

Andrei Dascalu 1 Reputation point
2024-01-17T16:20:47.09+00:00

Situation:

I have 2 AKS clusters in 2 separate resource groups each with its own VNET.

In one of the clusters I'm running some monitoring tools (Grafana, Prometheus, etc). Now I have the followng questions:

  • can I have Prometheus from cluster 1 scrape targets from cluster 2?
  • if not (or not easily feasible), how can I get Grafana from cluster 2 to access Prometheus from cluster 1 privately (without exposing it via public LB). I'm guessing I need to peer the VNETs, but unsure how to proceed beyond that.
  • how can I get promtail running in cluster 1 send data to Loki running in cluster 2? (I guess #2 may hold the answer to this as well)

Any pointers would be greatly appreciated

Thanks

Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
2,253 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Anveshreddy Nimmala 3,550 Reputation points Microsoft Vendor
    2024-01-19T03:53:44.3833333+00:00

    Hello Welcome to Microsoft Q&A, Thankyou for posting your query here. 1.Peer the VNET of Cluster 1 with the VNET of Cluster 2. This allows the two clusters to communicate privately. To peer the VNets, you need to follow these steps: i. In the Azure portal, go to the resource group that contains the VNet of cluster 1. ii. Select the VNet of cluster 1 and click on "Peerings" under "Settings". iii. Click on "Add" to add a new peering. iv. In the "Add peering" blade, enter a name for the peering and select the subscription and resource group that contains the VNet of cluster 2. Select the VNet of cluster 2 and click "OK" to create the peering. 2.After peering is created, you need to configure the Prometheus server running in cluster 1 to listen on a private IP address. You can do this by setting the --web.listen-address flag to the private IP address of the Prometheus server. 3.To send data from Promtail running in cluster 1 to Loki running in cluster 2, you can use the same VNet peering setup. You need to configure Promtail to send logs to the private IP address of Loki running in cluster 2. You can do this by setting the loki-url flag to the private IP address of Loki. Hope this answer helps you. please accept the answer for the sake of community, if the answer is helpful else post your query here in detail we can provide solution with having more details.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.