Use KQL Query in ARM template

Aakansha Verma 1 Reputation point
2022-04-20T13:11:18.667+00:00

@Sachin Kalia / @SwathiDhanwada-MSFT - I tried this solution provided by you https://learn.microsoft.com/en-us/answers/questions/379600/creating-dashboard-using-kusto-query-in-arm-templa.html. But its not working for me.

I'm trying to create a azure monitor workbook in App insight and have a ARM template for it, it have the KQL queries also. I want to parameterized it, specifically some parameters in my kql query. Using the above solution, i also used concat function, but its not working for me. Below is my code.
{
"contentVersion": "1.0.0.0",
"parameters": {
"workbookDisplayName": {
"type": "string",
"defaultValue": "x1",
"metadata": {
"description": "The friendly name for the workbook that is used in the Gallery or Saved List. This name must be unique within a resource group."
}
},
"workbookType": {
"type": "string",
"defaultValue": "workbook",
"metadata": {
"description": "The gallery that the workbook will been shown under. Supported values include workbook, tsg, etc. Usually, this is 'workbook'"
}
},
"workbookSourceId": {
"type": "string",
"defaultValue": "x2",
"metadata": {
"description": "Name of the app insight"
}
},
"workbookId": {
"type": "string",
"defaultValue": "[newGuid()]",
"metadata": {
"description": "The unique guid for this workbook instance"
}
},
"subscriptionId": {
"type": "string",
"defaultValue": "x3",
"metadata": {
"description": "Subscription id where the app insight resides"
}
},
"resourceGroupName": {
"type": "string",
"defaultValue": "x4",
"metadata": {
"description": "resource Group Name where the app insight resides"
}
},
"cloud_roleName": {
"type": "string",
"defaultValue": "x5",
"metadata": {
"description": "cloud_RoleName"
}
}
},
"variables": {
"workbookContent": {
"version": "Notebook/1.0",
"items": [
{
"type": 9,
"content": {
"version": "KqlParameterItem/1.0",
"parameters": [
{

                   "version": "KqlParameterItem/1.0",  
                   "name": "Apps",  
                   "type": 5,  
                   "isRequired": true,  
                   "multiSelect": true,  
                   "quote": "'",  
                   "delimiter": ",",  
                   "typeSettings": {  
                     "resourceTypeFilter": {  
                       "microsoft.insights/components": true  
                     },  
                     "additionalResourceOptions": [  
                       "value::all",  
                       "value::3"  
                     ]  
                   },  
                   "value": [  
                     "value::3"  
                   ]  
                 },  
                 {  

                   "version": "KqlParameterItem/1.0",  
                   "name": "TimeRange",  
                   "type": 4,  
                   "typeSettings": {  
                     "selectableValues": [  
                       {  
                         "durationMs": 300000,  
                         "createdTime": "2018-08-06T23:52:38.870Z",  
                         "isInitialTime": false,  
                         "grain": 1,  
                         "useDashboardTimeRange": false  
                       },  
                       {  
                         "durationMs": 900000,  
                         "createdTime": "2018-08-06T23:52:38.871Z",  
                         "isInitialTime": false,  
                         "grain": 1,  
                         "useDashboardTimeRange": false  
                       },  
                       {  
                         "durationMs": 1800000,  
                         "createdTime": "2018-08-06T23:52:38.871Z",  
                         "isInitialTime": false,  
                         "grain": 1,  
                         "useDashboardTimeRange": false  
                       },  
                       {  
                         "durationMs": 3600000,  
                         "createdTime": "2018-08-06T23:52:38.871Z",  
                         "isInitialTime": false,  
                         "grain": 1,  
                         "useDashboardTimeRange": false  
                       },  
                       {  
                         "durationMs": 14400000,  
                         "createdTime": "2018-08-06T23:52:38.871Z",  
                         "isInitialTime": false,  
                         "grain": 1,  
                         "useDashboardTimeRange": false  
                       },  
                       {  
                         "durationMs": 43200000,  
                         "createdTime": "2018-08-06T23:52:38.871Z",  
                         "isInitialTime": false,  
                         "grain": 1,  
                         "useDashboardTimeRange": false  
                       },  
                       {  
                         "durationMs": 86400000,  
                         "createdTime": "2018-08-06T23:52:38.871Z",  
                         "isInitialTime": false,  
                         "grain": 1,  
                         "useDashboardTimeRange": false  
                       },  
                       {  
                         "durationMs": 172800000,  
                         "createdTime": "2018-08-06T23:52:38.871Z",  
                         "isInitialTime": false,  
                         "grain": 1,  
                         "useDashboardTimeRange": false  
                       },  
                       {  
                         "durationMs": 259200000,  
                         "createdTime": "2018-08-06T23:52:38.871Z",  
                         "isInitialTime": false,  
                         "grain": 1,  
                         "useDashboardTimeRange": false  
                       },  
                       {  
                         "durationMs": 604800000,  
                         "createdTime": "2018-08-06T23:52:38.871Z",  
                         "isInitialTime": false,  
                         "grain": 1,  
                         "useDashboardTimeRange": false  
                       }  
                     ],  
                     "allowCustom": null  
                   },  
                   "value": {  
                     "durationMs": 14400000  
                   }  
                 },  
                 {  

                   "version": "KqlParameterItem/1.0",  
                   "name": "Operation",  
                   "type": 1,  
                   "isHiddenWhenLocked": true,  
                   "value": "{\"App\":\"\",\"Operation\":\"\"}"  
                 },  
                 {  

                   "version": "KqlParameterItem/1.0",  
                   "name": "Columns",  
                   "type": 2,  
                   "isRequired": true,  
                   "multiSelect": true,  
                   "quote": "",  
                   "delimiter": ",",  
                   "jsonData": "[[\"Mean\", \"Median\", \"p80\", \"p95\", \"p99\", \"Count\", \"Users\"]",  
                   "value": [  
                     "Mean",  
                     "Median",  
                     "p95",  
                     "Count"  
                   ]  
                 },  
                 {  

                   "version": "KqlParameterItem/1.0",  
                   "name": "In",  
                   "type": 2,  
                   "multiSelect": true,  
                   "quote": "'",  
                   "delimiter": ",",  
                   "query": "[concat('let cloud_RoleName = \"', parameters('cloud_roleName'), '\"; requests\r\n| where timestamp >= ago(7d)\r\n| where cloud_RoleName == cloud_roleName\r\n| summarize Count = count() by operation_Name\r\n| order by Count desc\r\n| project v = operation_Name, t = operation_Name, s = false\r\n| union (datatable(v:string, t:string, s:boolean)[\r\n'*', 'All Operations', true\r\n]))]",  
                   "typeSettings": {  
                     "additionalResourceOptions": []  
                   },  
                   "queryType": 0,  
                   "resourceType": "microsoft.insights/components"  
                 }  
               ],  
               "style": "pills",  
               "queryType": 0,  
               "resourceType": "microsoft.insights/components"  
             },  
             "name": "parameters - 1"  
           }         
         ],  
         "isLocked": false,  
         "fallbackResourceIds": [  
             "[resourceId(parameters('subscriptionId'), parameters('resourceGroupName'), 'microsoft.insights/components', parameters('workbookSourceId'))]"  
         ],  
         "fromTemplateId": "community-Workbooks/Performance/Performance Insights"  
       }  
     },  
     "resources": [  
       {  
         "name": "[parameters('workbookId')]",  
         "type": "microsoft.insights/workbooks",  
         "location": "[resourceGroup().location]",  
         "apiVersion": "2021-03-08",  
         "dependsOn": [],  
         "kind": "shared",  
         "properties": {  
           "displayName": "[parameters('workbookDisplayName')]",  
           "serializedData": "[string(variables('workbookContent'))]",  
           "version": "1.0",  
           "sourceId": "[resourceId(parameters('subscriptionId'), parameters('resourceGroupName'), 'microsoft.insights/components', parameters('workbookSourceId'))]",  
           "category": "[parameters('workbookType')]"  
         }  
       }  
     ],  
     "outputs": {  
       "workbookId": {  
         "type": "string",  
         "value": "[resourceId( 'microsoft.insights/workbooks', parameters('workbookId'))]"  
       }  
     },  
     "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#"  
   }  

The original query was -
"query": "requests\r\n| where timestamp >= ago(7d)\r\n| where cloud_RoleName == 'Placements'\r\n| summarize Count = count() by operation_Name\r\n| order by Count desc\r\n| project v = operation_Name, t = operation_Name, s = false\r\n| union (datatable(v:string, t:string, s:boolean)[\r\n'*', 'All Operations', true\r\n])",

to which i tried to parameterized to below-

"query": "[concat('let cloud_RoleName = \"', parameters('cloud_roleName'), '\"; requests\r\n| where timestamp >= ago(7d)\r\n| where cloud_RoleName == cloud_roleName\r\n| summarize Count = count() by operation_Name\r\n| order by Count desc\r\n| project v = operation_Name, t = operation_Name, s = false\r\n| union (datatable(v:string, t:string, s:boolean)[\r\n'*', 'All Operations', true\r\n]))]"

as i want to parameterized cloud_RoleName in the KQL query.

below is the error Im getting -

PS C:\> az deployment group create --resource-group co-wus2-amnoneshared-rg-t01 --name Performance-Analysis-Credentialing-7 --template-file .\workbooktemplate.json
{'code': 'InvalidTemplate', 'message': 'Deployment template validation failed: \'The template variable \'workbookContent\' is not valid: The language expression \'concat(\'let cloud_RoleName = "\', parameters(\'cloud_roleName\'), \'"; requests\r\n| where timestamp >= ago(7d)\r\n| where cloud_RoleName == cloud_roleName\r\n| summarize Count = count() by operation_Name\r\n| order by Count desc\r\n| project v = operation_Name, t = operation_Name, s = false\r\n| union (datatable(v:string, t:string, s:boolean)[\r\n\'\', \'All Operations\', true\r\n)\' is not valid: the string character \'\' at position \'333\' is not expected.. Please see https://aka.ms/arm-template-expressions for usage details.\'.', 'additionalInfo': [{'type': 'TemplateViolation', 'info': {'lineNumber': 55, 'linePosition': 26, 'path': 'properties.template.variables.workbookContent'}}]}

Azure Monitor
Azure Monitor
An Azure service that is used to collect, analyze, and act on telemetry data from Azure and on-premises environments.
3,645 questions
Azure DevTest Labs
Azure DevTest Labs
An Azure service that is used for provisioning development and test environments.
291 questions
Microsoft Security Microsoft Graph
0 comments No comments
{count} votes

2 answers

Sort by: Most helpful
  1. Alistair Ross 7,466 Reputation points Microsoft Employee
    2022-04-20T14:01:55.617+00:00

    Hello @Aakansha Verma ,

    Your issue is due to escaping characters within an ARM expression. The end of the query needs to look like this to escape the single quotes (Note the double single quotes)

    (datatable(v:string, t:string, s:boolean)[\r\n''*'', ''All Operations'', true\r\n])')]",  
    

    HOWEVER, I wouldn't concatenate a query this way, it adds more complexity and doesn't allow for it to be as dynamic. I would pass the ARM parameters into a Hidden workbook parameter (Parameter examples below) , and just call that parameter within the Workbook. This makes it far easier to design and scale. Furthermore when using Let statements, try to make the variable names different from the column names you are comparing, not just by case as it will make it confusing for you and others in the future when looking back at the code.

      {  
          "type": 9,  
          "content": {  
            "version": "KqlParameterItem/1.0",  
            "parameters": [  
              {  
                "id": "e466a430-a74e-4e3d-9bcc-6f037a1a38f8",  
                "version": "KqlParameterItem/1.0",  
                "name": "SelectedCloudRole",  
                "type": 1,  
                "isRequired": true,  
                "value": "[parameters('YOUR CLOUD ROLE NAME')] ",  
                "isHiddenWhenLocked": true  
              },  
              {  
                "id": "93f4579c-b2aa-490c-9be6-5ace64935854",  
                "version": "KqlParameterItem/1.0",  
                "name": "In",  
                "type": 2,  
                "multiSelect": true,  
                "quote": "'",  
                "delimiter": ",",  
                "query": "let selectedCloud_RoleName = \"{SelectedCloudRole}\"; \r\nrequests\r\n| where timestamp >= ago(7d)\r\n| where cloud_RoleName == selectedCloud_RoleName\r\n| summarize Count = count() by operation_Name\r\n| order by Count desc\r\n| project v = operation_Name, t = operation_Name, s = false\r\n| union (datatable(v:string, t:string, s:boolean)[\r\n'*', 'All Operations', true\r\n])",  
                "typeSettings": {  
                  "additionalResourceOptions": [],  
                  "showDefault": false  
                },  
                "timeContext": {  
                  "durationMs": 86400000  
                },  
                "queryType": 0,  
                "resourceType": "microsoft.insights/components"  
              }  
            ],  
            "style": "pills",  
            "queryType": 0,  
            "resourceType": "microsoft.insights/components"  
          },  
          "name": "parameters - 1"  
        }  
    
    0 comments No comments

  2. Aakansha Verma 1 Reputation point
    2022-04-20T19:47:28.273+00:00

    Thanks @Alistair Ross , I really appreciate your quick response and guidance. The above solution worked for me.

    Since I'm having more than one "items" under workbookContent. So I created a common "item" (line 6 in attachment) for having a hidden parameter, which can be consumed by any item underneath workbookContent as a parameter in their kql query. Also is it the right approach when using multiple let commands in kql query (line 184) .

    Pls check below code, let me if that makes sense.

    194862-arm-kqljson.txt

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.