Storage Account Routing & Azure DevOps Host Agent

Question

I have a StorageV2 account with diagnostic logs configured and it has the following access related properties:

{
    "allowSharedKeyAccess": true,
    "networkAcls": {
        "bypass": "AzureServices",
        "defaultAction": "Deny",
        "ipRules": [],
        "resourceAccessRules": [],
        "virtualNetworkRules": []
    },
    "publicNetworkAccess": "Enabled",
    "routingPreference": {
        "publishInternetEndpoints": true,
        "publishMicrosoftEndpoints": false,
        "routingChoice": "MicrosoftRouting"
    }
}

I then have a script that uploads a directory to a file share that uses the primary access key and the Internet facing endpoint:

# code above this omitted as not required
$Env:AZURE_STORAGE_ACCOUNT = $StorageAccount

# retrieve the primary access key
$Env:AZURE_STORAGE_KEY = az storage account keys list `
    --account-name $StorageAccount `
    --query "[?keyName=='key1'].value" `
    --output $AzOutput

# retrieve the internet facing file endpoint
$Env:AZURE_STORAGE_SERVICE_ENDPOINT = az storage account show `
    --name $StorageAccount `
    --query primaryEndpoints.internetEndpoints.file `
    --output $AzOutput

# retrieve public ip
$publicIp = Invoke-RestMethod https://ipinfo.io/json | Select-Object -exp ip

# add ip to allow list
# custom function that adds a sleep on the end as there's no `az storage wait` command
Set-StorageNetworkRule `
    -StorageAccount $StorageAccount `
    -ResourceGroup $ResourceGroup `
    -IpAddress $publicIp `
    -Action add `
    -WaitSeconds $WaitSeconds `
    -AzOutput $AzOutput

# upload directory to file share
# --account-name, --access-key, and --file-endpoint not required as environment variables will be used
az storage file upload-batch `
    --destination $DestinationShare `
    --destination-path $DestinationPath `
    --source $SourcePath `
    --output $AzOutput

• Running it from my local console
  • Network rule is added with my public IP address
  • Upload completes successfully
  • network.client.ip in the logs matches my public IP address
• Running it from an Azure DevOps pipeline inside a Microsoft Hosted Agent:
  • Network rule is added with its public IP address
  • ~90% of the time the upload fails with AuthorizationError
  • network.client.ip:
  • When it fails: RFC 1918 address in the 10.1.0.0 range
  • When it succeeds: matches its public IP address

To confirm that the recorded client IP the issue, the upload completes 100% of the time if I remove the network restrictions and enable public access from all networks.

My understanding is that, even with Microsoft routing selected, the request should originate from the client's public facing IP address if the Interface facing endpoint has been configured and is being used.

As I've been unable to find any official documentation on the behaviour of routing options on network.client.ip, could someone please confirm what is required for it to return the actual client IP and thus work with network rules as expected?

Could the underlying cause be due to the Microsoft host agent being spun up in the same location and therefore will always use the Microsoft routed IP address?

Note that using a self-hosted agent and making use of the private endpoint is not currently an option.

Answer 1

@Anonymous Firstly, Apologies for the delay response,

If you have selected to enable for certain networks in firewall settings then you must access the file share from that network which is allowed by the firewall

I wish to engage with you offline for a closer look and provide a quick and specialized assistance, please send an email with subject line “Attn:subm” to AzCommunity[at]Microsoft[dot]com referencing this thread and the Azure subscription ID, I will follow-up with you.  Once again, apologies for any inconvenience with this issue.
Thanks for your patience and co-operation.

---

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.