Internal Server Error for deploying Vnet Flow Logs with Traffic Analytics

Question

Internal Server Error for deploying Vnet Flow Logs with Traffic Analytics

Jay 86

Trying to deploy the new Azure VNET Flow Logs with traffic analytics, however, not able to succeed so far. I have attempted azure policy, azure cli,portal manually, and REST API all are failing with the internal server error


{

  "name": "vnetflowlog",

  "properties": {

    "targetResourceId": "...../vnet-01",

    "storageId": "...flowsa01",

    "enabled": true,

    "flowAnalyticsConfiguration": {

      "networkWatcherFlowAnalyticsConfiguration": {

        "enabled": true,

        "workspaceId": "310dfd4a-1137-4a05-8130-abde",

        "workspaceRegion": "eastus2",

        "workspaceResourceId": ".../abcdef",

        "trafficAnalyticsInterval": 60

      }

    },

    "retentionPolicy": {

      "days": 7,

      "enabled": true

    },

    "format": {

      "type": "JSON",

      "version": 2

    }

  },

  "type": "Microsoft.Network/networkWatchers/flowLogs",

  "location": "eastus2",

}

It failed with the below error always

  "content": {

    "status": "Failed",

    "error": {

      "code": "InternalServerError",

      "message": "An error occurred.",

      "details": []

    }

  }

and makes the provisioningState to Failure If i just removed or make the networkWatcherFlowAnalyticsConfiguration.enabled to false its working fine. So, is there anyone able to successfully deploy the vent flow logs with traffic analytics enabled without having such an error? Let me know If I'm making any mistake in my deployment

KapilAnanth 49,866 Reputation points Moderator

2024-09-09T05:23:55.5233333+00:00

@Jay ,

This looks like an issue specific to your environment.

To troubleshoot further, we will need a specialized 1:1 session, where a support engineer can have a screen share session and check the backend logs to pinpoint the issue.

If you have a support plan you may file a support ticket, else please do let us know, we will try and help you get a one-time free technical support.

Cheers,

Kapil
KapilAnanth 49,866 Reputation points Moderator

2024-11-25T15:34:21.3433333+00:00

@Jay ,

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this!

Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others", I have summarized our discussion into an answer in case you'd like to "Accept" the answer.

RCA :

After a deeper investigation I have found that the reason for this error is due the policy enforcement that we deployed for tag compliance

So whenever we create a vnet flow logs with traffic analytics its internaly creating a new resource called dataCollectionEndpoints . This resource is not having the proper tags which getting the cause of the entier flow log to fail

Thank you again for your time and patience throughout this issue.

Cheers,

Kapil.

Please remember to "Accept Answer" if any answer/reply helped, so that others in the community facing similar issues can easily find the solution.

2 answers

Your answer

KapilAnanth 49,866 Reputation points Moderator

2024-09-09T05:23:55.5233333+00:00

@Jay ,

This looks like an issue specific to your environment.

To troubleshoot further, we will need a specialized 1:1 session, where a support engineer can have a screen share session and check the backend logs to pinpoint the issue.

If you have a support plan you may file a support ticket, else please do let us know, we will try and help you get a one-time free technical support.

Cheers,

Kapil
KapilAnanth 49,866 Reputation points Moderator

2024-11-25T15:34:21.3433333+00:00

@Jay ,

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this!

Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others", I have summarized our discussion into an answer in case you'd like to "Accept" the answer.

RCA :

After a deeper investigation I have found that the reason for this error is due the policy enforcement that we deployed for tag compliance

So whenever we create a vnet flow logs with traffic analytics its internaly creating a new resource called dataCollectionEndpoints . This resource is not having the proper tags which getting the cause of the entier flow log to fail

Thank you again for your time and patience throughout this issue.

Cheers,

Kapil.

Please remember to "Accept Answer" if any answer/reply helped, so that others in the community facing similar issues can easily find the solution.

Answer 1

After a deeper investigation I have found that the reason for this error is due the policy enforcement that we deployed for tag compliance

So whenever we create a vnet flow logs with traffic analytics its internaly creating two new resources called dataCollectionEndpoints and dataCollectionRules. This resource is not having the proper tags which getting the cause of the entire flow log to fail .

Update

After i tried to pass the tags for vnet flow logs but still it getting failed this is because the tag that i passed during vnetflow logs is not inherited by the 2 resources (dataCollectionRules and dataCollectionEndpoints) so the only way this to work by updating the policy definition to allow these resources with specific naming pattern as the names always starts with NWTA-* i hope ms will not change this naming pattern in the backend :)

Sample Policy Def

{
  "mode": "Indexed",
  "policyRule": {
    "if": {
      "allOf": [
        {
          "anyOf": [
            {
              "allOf": [
							{
                "field": "type",
                "in": [
                  "Microsoft.Insights/dataCollectionEndpoints",
                  "Microsoft.Insights/dataCollectionRules"
                ]
              },
                {
                  "field": "name",
                  "notContains": "NWTA-"
                },
                {
                  "field": "[concat('tags[', parameters('tagName'), ']')]",
                  "exists": "false"
                }
              ]
            },
            {
              "allOf": [
              {
                "field": "type",
                "notIn": [
                  "Microsoft.Insights/dataCollectionEndpoints",
                  "Microsoft.Insights/dataCollectionRules"
                ]
              },
                {
                  "field": "[concat('tags[', parameters('tagName'), ']')]",
                  "exists": "false"
                }
              ]
            }
          ]
        }
      ]
    },
    "then": {
      "effect": "deny"
    }
  }

Answer 2

KapilAnanth 49,866 Moderator

@Jay ,

Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well.

Unfortunately, I am unable to repo the issue.

I just did a Lab (via Portal) and I was able to create a VNET Flow Log with Traffic Analytics enabled.
Can you please try it again ?
- If you are using regular portal, try preview portal : https://portal.azure.com/?preview=yes
- If you are using preview portal, try normal portal : https://portal.azure.com/?preview=no

If the issue persists,

Did you try with different user accounts?
Did you try to use a different VNET/Subnet (for testing)?
May I ask the region you are trying to create a VNET Flow Log
- Can you try to use a different region(for testing)?

Cheers,

Kapil

Jay 86 Reputation points

2024-08-13T11:49:13.8766667+00:00
Hi @KapilAnanth , Same error

Did you try with different user accounts? - Yes

Did you try to use a different VNET/Subnet (for testing)? - Yes

May I ask the region you are trying to create a VNET Flow Log -EASTUS2

Can you try to use a different region(for testing)? - We currently dont have any other region we are allowed to implement this only on eastus2 region
KapilAnanth 49,866 Reputation points Moderator

2024-08-15T06:30:15.94+00:00
@Jay

Hope you tried the above recommendations with Portal

I was able to successfully deploy Flow Logs with TA enabled for East US 2

As next steps,

#1

From activity log , please check the entire JSON for any errors

#2

First deploy the Flow Logs and let is succeed without traffic analytics

And then try to enable traffic analytics and see what error you are getting

If it fails,

Please share the entire error message from Notifications

Also, share the error message from activity log as well

Cheers,

Kapil

Jay 86

@KapilAnanth

From the activity log

"properties": {
        "statusCode": "OK",
        "statusMessage": "\"{\\\"provisioningState\\\":\\\"Failed\\\",\\\"error\\\":\\\"{\\\\\\\"code\\\\\\\":\\\\\\\"DeploymentFailed\\\\\\\",\\\\\\\"target\\\\\\\":\\\\\\\"/subscriptions/<removed>/resourceGroups/networkwatcherrg/providers/Microsoft.Resources/deployments/flowlogDeployment-34txnsumyldpm\\\\\\\",\\\\\\\"message\\\\\\\":\\\\\\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\\\\\",\\\\\\\"details\\\\\\\":[{\\\\\\\"code\\\\\\\":\\\\\\\"ResourceDeploymentFailure\\\\\\\",\\\\\\\"target\\\\\\\":\\\\\\\"/subscriptions/<removed>/resourceGroups/networkwatcherrg/providers/Microsoft.Network/networkWatchers/NetworkWatcher_eastus2/flowLogs/demo-vnet-01-demo-network-rg-01-flowlog\\\\\\\",\\\\\\\"message\\\\\\\":\\\\\\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\\\\\",\\\\\\\"details\\\\\\\":[{\\\\\\\"code\\\\\\\":\\\\\\\"InternalServerError\\\\\\\",\\\\\\\"message\\\\\\\":\\\\\\\"An error occurred.\\\\\\\",\\\\\\\"details\\\\\\\":[]}]}]}\\\"}\"",

"submissionTimestamp": "2024-09-08T14:40:32Z",
    "subscriptionId": "<removed>",
    "tenantId": "<removed>",
    "properties": {
        "statusMessage": "{\"status\":\"Failed\",\"error\":{\"code\":\"ResourceOperationFailure\",\"message\":\"The resource operation completed with terminal provisioning state 'Failed'.\",\"details\":[{\"code\":\"InternalServerError\",\"message\":\"An error occurred.\",\"details\":[]}]}}",
        "eventCategory": "Administrative",
        "entity": "/subscriptions/<removed>/resourcegroups/networkwatcherrg/providers/Microsoft.Network/networkWatchers/NetworkWatcher_eastus2/flowLogs/demo-vnet-01-demo-network-rg-01-flowlog",
        "message": "Microsoft.Network/networkWatchers/flowLogs/write",
        
    }

2 I was able to deploy without traffic anayltics but on top of that i cannot able to manually click on the portal to enable traffic analytics. If i tried with rest api then i get the same error as the above activity log

Share via

Internal Server Error for deploying Vnet Flow Logs with Traffic Analytics

2 answers

Your answer