Azure pipelines - agent ip changed during single job

Szymon 0

Hello,

I have following (simplified) steps in pipeline:

Read agent IP;
Add agent ip to azure cosmosdb firewall;
Run tests which read data from cosmosdb.

Today, second time this week (previously this happened on monday, and I have not seen such scenario for like 1 year) I received error that in step 3 IP from which request to azure comosdb was orignated is different than ip read in step 1. All above steps are under same job (so same agent is used). How it is possible that agent changed its ip during single agent runtime? I think that should be reported to MS but first wanted to know if anyone else thinks/knows otherwise.

ShaktiSingh-MSFT 14,406 Reputation points Microsoft Employee

2023-07-21T14:02:03.96+00:00

H iSzymon,

•

Welcome to Microsoft Q&A forum and thanks for using Azure Services.

As I understand, you want agent ip changed in single job.

Could you please elaborate more on the steps you are performing and the pipelines you are creating so that we can assist you.

Thank you
Szymon 0 Reputation points

2023-07-21T15:41:09.7066667+00:00
Hi Shakti,

Actually quite opposite, it has changed and as far as I know during single agent runtime it shouldn't, because of that ip change I was not able to access cosmos db as I am adding agent ip to cosmos db firewall, so I have read the ip, added it to firewall (so agent could access database), but as it has changed during runtime I have received error:

Response status code does not indicate success: Forbidden (403); Substatus: 0; ActivityId: 3081666b-e17f-42cc-82e0-13601d0d46b1; Reason: (Request originated from IP 40.77.83.28 through public internet. This is blocked by your Cosmos DB account firewall settings. More info: https://aka.ms/cosmosdb-tsg-forbidden

While the ip which was read in step 1 was equal to:

40.77.83.20
ShaktiSingh-MSFT 14,406 Reputation points Microsoft Employee

2023-07-24T08:25:33.64+00:00

Hi Szymon,

This is strange and not expected. I would recommend you to please file a support ticket for deeper investigation and in case if you don't have a support plan, do let us know here so that we can check on other options to unblock you. Thanks

Oly Dungey 0

I have the same problem - I am pushing to an Azure Repository from an Azure Devops pipeline. To do so I get the agent to whitelist itself first. Every 1 in 30 times or so it fails as the external IP address of the agent appears to change between tasks. I have tried various whatsmyip style websites with the same result. Here is the pipeline code:

- task: AzureCLI@2
    displayName: Add agent IP to ACR firewall whitelist
    inputs:
      azureSubscription: $(azureSubscription)
      scriptType: bash
      scriptLocation: inlineScript
      inlineScript: |
        agentIP=$(curl --show-error --no-progress-meter https://ipinfo.io/ip)

        function isIpWhitelisted() {
          az acr network-rule list --name $(containerRegistryName) | grep -q "$agentIP"
        }

        # Has this agent already been whitelisted?
        if isIpWhitelisted; then
            echo "Agent IP [$agentIP] already whitelisted in ACR [$(containerRegistryName)] firewall"
            echo "##vso[task.setvariable variable=agentWhitelistedForACR]true"
            exit 0
          fi

        echo "Adding agent IP [$agentIP] to ACR [$(containerRegistryName)] firewall whitelist"
        if ! az acr network-rule add --name $(containerRegistryName) --ip-address $agentIP; then
          echo "Failed to add [network-rule] to ACR [$(containerRegistryName)]"
          echo "##vso[task.setvariable variable=agentWhitelistedForACR]false"
          exit 1
        fi

        for i in {1..60}; do
          echo "Attempt [$i] to confirm IP [$agentIP] whitelisted in ACR [$(containerRegistryName)] firewall"
          if isIpWhitelisted; then
            echo "Agent IP [$agentIP] confirmed whitelisted in ACR [$(containerRegistryName)] firewall"
            echo "##vso[task.setvariable variable=agentWhitelistedForACR]true"
            exit 0
          fi
          sleep 1
        done

        echo "Failed to verify whitelisting of Agent IP [$agentIP] in ACR [$(containerRegistryName)] firewall after multiple retries"
        echo "##vso[task.setvariable variable=agentWhitelistedForACR]false"
        exit 1

  - task: DockerCompose@0
    displayName: Docker Compose Push
    inputs:
      containerregistrytype: Azure Container Registry
      azureSubscription: $(azureSubscription)
      azureContainerRegistry: ***REDACTED***
      dockerComposeFile: $(Build.Repository.LocalPath)/docker-compose.yml
      action: 'Push services'
      includeSourceTags: true
      includeLatestTag: true
      additionalImageTags: |
          $(FullSemVer)

It is driving me crazy, I have tried all sorts of different approaches and always end up with the same issue.

Adeel Qayyum 0 Reputation points

2023-11-17T15:40:03.4566667+00:00

@ShaktiSingh-MSFT We have the same problem, was there any solution provided for this ?
ShaktiSingh-MSFT 14,406 Reputation points Microsoft Employee

2023-11-20T08:32:14.21+00:00

Hi Adeel Qayyum •, Oly Dungey •,

As suggested above, please file a support ticket for deeper investigation and in case if you don't have a support plan, do let us know here so that we can check on other options to unblock you. Thanks
Adeel Qayyum 0 Reputation points

2023-11-30T14:20:45.5166667+00:00

@ShaktiSingh-MSFT don't think we have a support plan. Is there another way to log a ticket ?

1 answer

ShaktiSingh-MSFT 14,406 Reputation points Microsoft Employee

2023-12-01T03:57:51.0866667+00:00

Hi Adeel Qayyum •,

The query posted by you seems to have issue in Azure DevOps which is not supported in the Q&A forum. I would request you post your query in the Azure DevOps forum

This will assist you with a faster reply to your query.

Hope this helps. If the issue lies in Azure Cosmos DB, then I can help you with support ticket.

Regards,

@ShaktiSingh-MSFT
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Share via

Azure pipelines - agent ip changed during single job

1 answer