Azure Policy to deploy VM application is success when run on our Resource group, but app is not installed

rajesh.john 0

We are attempting to install software agents on our VM's in Azure (looking for something similar to aws systems manager and state manager)

Azure Policy that has VM application defined on it runs fine and also completes with success, but application is not installed on it...

Where can i find the logs for this azure policy remediations ?

2 answers

Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-08T19:37:05.25+00:00

Hi rajesh.john,
Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

To troubleshoot the installation of software agents on your Azure VMs using Azure Policy, you can find logs related to policy remediations in several places within the Azure portal. Here’s where to look for logs and troubleshoot your Azure Policy remediation tasks:
Azure Portal - Policy Compliance:

Navigate to the Azure Portal.
Go to Policy and select Assignments.
Locate the specific policy assignment related to your VM applications.
Click on the assignment to view its compliance state. This will show which resources are compliant or non-compliant.

Activity Log:
In the Azure Portal, go to Monitor > Activity Log.
Filter the logs by resource type (e.g., Virtual Machines) and the specific time frame during which you executed the policy.
Look for events related to policy evaluations and remediation actions.

Guest Configuration Logs (for Guest Configuration Policies):

If using Guest Configuration policies, check the logs on the VM itself. The Guest Configuration extension logs are typically located at: C:\Windows\Logs\Azure\GuestConfiguration

These logs provide insights into whether the Guest Configuration agent was able to apply the desired settings or install applications.
Azure Automation Runbook Logs:
If you are using Azure Automation runbooks to handle installations based on policy compliance, check the output of those runbooks in the Azure Automation account.

Go to Automation Accounts, select your account, then navigate to Jobs under Process Automation to see details of each runbook execution.

For better understanding please refer attached link: https://techcommunity.microsoft.com/blog/coreinfrastructureandsecurityblog/software-installation-using-machine-configuration-and-azure-policy/3695636

If you have any further queries, do let us know. If the Answer is helpful, please click "Accept Answer".
Please sign in to rate this answer.
Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-11T16:19:04.5966667+00:00

Hi rajesh.john,
We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion.
Thank You.!

rajesh.john 0 Reputation points

2024-11-12T14:23:48.4066667+00:00

Thanks for details on finding logs for these various resources used in this Azure policy..

I am still not getting this application installed on VM via Azure policy, does it help if i share my azure policy here?

My goal is to install this application via azure policy on VMs when this appllication is not found on our VMs

Regards

Rajesh

Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-12T17:55:19.7466667+00:00

Hi rajesh.john,
Thanks for replying back,
To install an application on Azure Virtual Machines (VMs) when it is not found, you can utilize Azure Policy, specifically the DeployIfNotExists policy effect. This approach automates the installation process and ensures compliance across your VMs. Here’s a structured guide on how to achieve this:

Steps to Install Applications via Azure Policy

1.Create an Azure Policy Definition

You need to define a policy that checks for the existence of the application on your VMs. The policy should specify the application you want to ensure is installed.

Policy Type: Use DeployIfNotExists, which deploys the application if it is not already present.

Target Resource Type: Specify Microsoft.Compute/virtualMachines as the resource type.

Here’s an example of what your policy definition might look like in JSON format:

json

{

"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",

"contentVersion": "1.0.0.0",

"parameters": {

"vmName": { "type": "string" }

},

"resources": [

{ "type": "Microsoft.Compute/virtualMachines/VMapplications", "name": "[concat(parameters('vmName'), '/YourApplicationName')]", "properties": { // Application properties here } }

],

"effect": "DeployIfNotExists"

}

Assign the Policy

Once your policy is defined, you need to assign it to the appropriate scope, such as a resource group or subscription. During assignment, ensure that the managed identity used has sufficient permissions to deploy applications.

Use Guest Configuration

To audit compliance and trigger installations, leverage Azure’s Guest Configuration feature. This allows you to check if specific software is installed on your VMs:

Create a Guest Configuration policy that audits for the application.

Set up an Event Grid subscription to listen for state change events related to compliance.

Automation with Azure Automation Runbooks

To automate the installation process when a VM is found non-compliant:

Create an Azure Automation runbook that will execute a script to install the application.

Use PowerShell commands within the runbook to perform the installation.

For instance, you can use:

powershell

Invoke-AzVMRunCommand -ResourceGroupName 'YourResourceGroup' -VMName 'YourVM' -CommandId 'RunPowerShellScript' -ScriptPath 'path/to/your/install-script.ps1'

Remediation Task

For existing VMs that may not have the application installed, create a remediation task:

Navigate to your policy assignment in Azure Policy.

Select Create Remediation Task and choose the relevant policy.

This task will evaluate existing VMs and trigger the installation of the application where necessary.

Monitoring Compliance

After implementing these steps, monitor compliance through the Azure portal. The Guest Configuration service will report back on whether each VM is compliant with the policy.

For better understand please go through attached referral link: https://techcommunity.microsoft.com/blog/coreinfrastructureandsecurityblog/software-installation-using-azure-policy-state-change-events/2420515

If you required anything, please let us know. Thank you.

Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-13T16:32:19.1466667+00:00

Hi rajesh.john,
We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion.

rajesh.john 0 Reputation points

2024-11-14T15:14:04.1566667+00:00

Hello,

I am attempting to install splunk forwarder agent on windows VMs in our resource group and i have created gallery object and uploaded binaries to azure blob with install script and getting the error below.

Can some check my policy (found below error message)

error:

"{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.","details":[{"code":"BadRequest","message":"The gallery application reference id provided, '/subscriptions/e22d59e5-689e-4700-a276-a7f187f2471a/resourceGroups/SharedServices/providers/Microsoft.Compute/galleries/splunk', is invalid."}]}"

Azure policy:

{ "properties": { "displayName": "splunk", "policyType": "Custom", "mode": "All", "description": "Nov 14 @ 9:44 AM", "metadata": { "createdBy": "94c0440d-ab86-4e89-b1fe-4e2a16efb714", "createdOn": "2024-11-12T21:37:33.7588109Z", "updatedBy": "94c0440d-ab86-4e89-b1fe-4e2a16efb714", "updatedOn": "2024-11-14T14:44:31.433264Z" }, "version": "1.0.0", "parameters": {}, "policyRule": { "if": { "allOf": [ { "field": "type", "equals": "Microsoft.Compute/virtualMachines" }, { "field": "Microsoft.Compute/virtualMachines/storageProfile.osDisk.osType", "equals": "Windows" } ] }, "then": { "effect": "deployIfNotExists", "details": { "type": "Microsoft.Compute/virtualMachines", "name": "[field('name')]", "existenceCondition": { "allOf": [ { "count": { "field": "Microsoft.Compute/virtualMachines/applicationProfile.galleryApplications[*]", "where": { "field": "Microsoft.Compute/virtualMachines/applicationProfile.galleryApplications[*].packageReferenceId", "equals": "/subscriptions/e22d59e5-689e-4700-a276-a7f187f2471a/resourceGroups/SharedServices/providers/Microsoft.Compute/galleries/splunk" } }, "greater": 0 } ] }, "roleDefinitionIds": [ "/providers/Microsoft.Authorization/roleDefinitions/8e3af657-a8ff-443c-a75c-2fe8c4bcb635" ], "deployment": { "properties": { "mode": "incremental", "template": { "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json", "contentVersion": "1.0.0.0", "parameters": { "vmName": { "type": "string" }, "location": { "type": "string" } }, "resources": [ { "apiVersion": "2021-07-01", "type": "Microsoft.Compute/virtualMachines/VMapplications", "name": "[concat(parameters('vmName'),'/splunk')]", "location": "[parameters('location')]", "properties": { "packageReferenceId": "/subscriptions/e22d59e5-689e-4700-a276-a7f187f2471a/resourceGroups/SharedServices/providers/Microsoft.Compute/galleries/splunk" } } ] }, "parameters": { "vmName": { "value": "[field('name')]" }, "location": { "value": "[field('location')]" } } } } } } }, "versions": [ "1.0.0" ] }, "id": "/subscriptions/e22d59e5-689e-4700-a276-a7f187f2471a/providers/Microsoft.Authorization/policyDefinitions/b5ea1804-2752-422c-96d5-544e35103cc7", "type": "Microsoft.Authorization/policyDefinitions", "name": "b5ea1804-2752-422c-96d5-544e35103cc7", "systemData": { "createdBy": "rajesh.john@gordianfedcloud.onmicrosoft.com", "createdByType": "User", "createdAt": "2024-11-12T21:37:33.7366502Z", "lastModifiedBy": "rajesh.john@gordianfedcloud.onmicrosoft.com", "lastModifiedByType": "User", "lastModifiedAt": "2024-11-14T14:44:31.4166319Z" } }

Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-14T19:41:44.4266667+00:00

Hi rajesh.john,
Thanks for replying back.
To address the error you're encountering while trying to install the Splunk Forwarder agent on Windows VMs in Azure, let's analyze the provided error message and your Azure policy.

Error Overview

The error message indicates that the deployment failed due to an invalid gallery application reference ID:

{ "code": "BadRequest", "message": "The gallery application reference id provided, '/subscriptions/e22d59e5-689e-4700-a276-a7f187f2471a/resourceGroups/SharedServices/providers/Microsoft.Compute/galleries/splunk', is invalid." }

This suggests that the specified gallery application ID does not exist or is incorrectly formatted.

Possible Causes and Solutions

Verify Gallery Application ID:

Check if the gallery application with the ID /subscriptions/e22d59e5-689e-4700-a276-a7f187f2471a/resourceGroups/SharedServices/providers/Microsoft.Compute/galleries/splunk exists in your Azure subscription. You can do this via the Azure portal or using Azure CLI:

az sig show --gallery-name splunk --resource-group SharedServices

Ensure that the gallery is correctly created and that it contains the necessary images.

Policy Configuration:

Review your Azure policy definition to ensure it correctly references the gallery application. The policy should allow for deployments that utilize this gallery. If there are restrictions in place, you may need to adjust them.

The relevant part of your policy checks for the existence of a gallery application:

{ "count": { "field": "Microsoft.Compute/virtualMachines/applicationProfile.galleryApplications[*]", "where": { "field": "Microsoft.Compute/virtualMachines/applicationProfile.galleryApplications[*].packageReferenceId", "equals": "/subscriptions/e22d59e5-689e-4700-a276-a7f187f2471a/resourceGroups/SharedServices/providers/Microsoft.Compute/galleries/splunk" } } }

Make sure that this condition aligns with how you're deploying the VM and that it matches the actual resources present.

Deployment Permissions:

Ensure that you have adequate permissions to deploy resources using the specified gallery application. Lack of permissions could also lead to deployment failures.

Check for Policy Violations:

If there are existing policies that restrict certain actions or resource types, they may be blocking your deployment. You can check for policy violations in the Azure portal under Policy > Assignments.

To view detailed logs of failed deployments, navigate to your resource group in the Azure portal, select Deployments, and review any failed operations for more context.

Correcting Template Issues:

If you are using an ARM template, ensure that it adheres to the correct schema and includes all necessary parameters and configurations. The schema for ARM templates should be up-to-date:

"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#"

IF you required anything please let me know.

Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-18T16:59:50.5666667+00:00

Hi rajesh.john,
We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion. Thank You...!

Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-19T19:25:28.6966667+00:00

Hi rajesh.john,
Just want to check is that issue resolved or else if you need any help, please let us know we will always help as you needed.! if you had chance please review once my previous comment. Thank you

rajesh.john 0 Reputation points

2024-11-19T19:50:55.1+00:00

I got the azure policy completed, do you know how often does a custom azure policy run to remediate on azure resources ?

Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-19T21:35:19.82+00:00

Azure Policy operates through a structured evaluation and remediation process for Azure resources. Here’s how often custom Azure policies run to remediate resources:

Policy Evaluation Frequency

Regular Compliance Evaluation: Azure Policy conducts a standard compliance evaluation cycle once every 24 hours. During this cycle, all resources are assessed against the assigned policies to determine if they remain compliant or have drifted into a non-compliant state5.

Event-Driven Evaluations: In addition to the daily evaluations, resources are evaluated at specific events:

When a resource is created or updated.

When a policy or initiative is newly assigned or updated to a scope5.

Remediation Tasks

Initial Remediation: When you create a remediation task, it runs immediately after its creation to correct any non-compliant resources identified during the evaluation cycle3.

Subsequent Remediation: If resources become non-compliant again after the initial remediation, a new remediation task must be created. This means that remediation tasks do not run automatically after the first execution; they need to be manually triggered for subsequent compliance issuesAzure Policy operates through a structured evaluation and remediation process for Azure resources. Here’s how often custom Azure policies run to remediate resources: Policy Evaluation Frequency

Regular Compliance Evaluation: Azure Policy conducts a standard compliance evaluation cycle once every 24 hours. During this cycle, all resources are assessed against the assigned policies to determine if they remain compliant or have drifted into a non-compliant state5.

Event-Driven Evaluations: In addition to the daily evaluations, resources are evaluated at specific events:

When a resource is created or updated.
- When a policy or initiative is newly assigned or updated to a scope[5](https://learn.microsoft.com/en-us/azure/governance/policy/overview?WT.mc_id=modinfra-69845-socuff).

Remediation Tasks

Initial Remediation: When you create a remediation task, it runs immediately after its creation to correct any non-compliant resources identified during the evaluation cycle3.

Subsequent Remediation: If resources become non-compliant again after the initial remediation, a new remediation task must be created. This means that remediation tasks do not run automatically after the first execution; they need to be manually triggered for subsequent compliance issues.

Please let me know if any thing required

Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-20T17:03:33.31+00:00

Hi rajesh.john,
We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion. Thank You...!

rajesh.john 0 Reputation points

2024-11-20T17:18:13.3633333+00:00

Hello

I have created an policy to install splunk agent on our resource group VMs.

The current solution i have is using these, gallery object, vm application definition, application version & azure custom policy

I have question on how can we use custom VM application extensions to keep our splunk agents updated ?

Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-20T18:52:00.5833333+00:00

Thanks for replying rajesh.john
You can use custom VM application extensions to keep your Splunk agents updated on your Azure VMs.

To do this, you can follow these steps:

Create a custom VM application extension that installs the latest version of the Splunk agent on your Azure VMs. You can use a script or a custom executable to perform the installation.

Upload the custom VM application extension to your Azure subscription. You can do this by creating a new VM extension in the Azure portal and uploading the extension package.

Add the custom VM application extension to your Azure VMs. You can do this by selecting the VM in the Azure portal, clicking on "Extensions" in the left-hand menu, and then clicking on "Add". Select the custom VM application extension from the list of available extensions and follow the prompts to install it on the VM.

Configure the custom VM application extension to run on a schedule. You can do this by configuring a scheduled task in the extension script or executable.

Once the custom VM application extension is installed and configured, it will automatically update the Splunk agent on your Azure VMs on a schedule that you specify. This will help ensure that your Splunk agents are always up-to-date and running the latest version.

I hope this helps! Let me know if you have any further questions.

Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-21T16:39:52.5633333+00:00

Hi rajesh.john,
We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion. Thank You...!

Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-22T17:12:28.5133333+00:00

Hi rajesh.john,
Just want to check is that issue resolved or else if you need any help, please let us know we will always help as you needed.! if you had chance please review once my previous comment Thank you.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
rajesh.john 0 Reputation points

2024-11-22T18:07:10.1033333+00:00

Hello

the issue is not resolved yet, these are my progress on this effort

I am able to create VMAppExtension using Resource Manager template and also able to deploy them to my VM. After deployment i am able to see the VMAppExtension listed under Extension on my VM with Status (unavailable)..

But, the splunk agent is not installed on the VM, need to know what else should we check for this agent install via VM extensions
Please sign in to rate this answer.
Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-25T22:35:29.2566667+00:00

Hi rajesh.john
To troubleshoot the issue of the Splunk agent not being installed on your Azure VM despite the VMAppExtension showing as "unavailable," consider the following steps:

Check Azure VM Agent Status

Verify the Azure VM Agent: Ensure that the Azure VM Agent is installed and running on your VM. This agent is essential for managing extensions and must be in a "Ready" state to function correctly. You can check its status in the Azure portal or by looking at the logs located at C:\WindowsAzure\Logs\WaAppAgent.log on your VM

Review Extension Logs

Inspect Extension Logs: Check the logs specific to the Splunk extension, which can typically be found in:

C:\WindowsAzure\Logs\Plugins\<ExtensionName>\<Version>\CommandExecution.log

C:\Packages\Plugins\<ExtensionName>\<Version>\Status\status.json

These logs will provide detailed information about what went wrong during the installation process

Common Issues and Solutions

Existing Failed Extensions: If there are any existing extensions with a failed provisioning state, new extensions may fail to install. Remove any failed extensions using PowerShell or the Azure portal before attempting to redeploy

Timeouts and Resource Constraints: Extensions have a limited time to run (usually 20 minutes). If your VM is low on resources or if other processes are consuming significant resources, it may lead to timeouts. Ensure your VM has adequate resources available during installation1 4.

Network and Dependency Requirements: Some extensions require network access to download packages or communicate with external services. Ensure that your VM can access necessary endpoints, such as Azure Storage or package repositories1 4.

Rerun the Extension

Remove and Redeploy: If all else fails, consider removing the Splunk extension and redeploying it. Use PowerShell commands like:
powershell Remove-AzVMExtension -ResourceGroupName "<ResourceGroup>" -VMName "<VMName>" -Name "<ExtensionName>"
After removal, you can redeploy the extension using your ARM templateTo troubleshoot the issue of the Splunk agent not being installed on your Azure VM despite the VMAppExtension showing as "unavailable," consider the following steps: Check Azure VM Agent Status

Verify the Azure VM Agent: Ensure that the Azure VM Agent is installed and running on your VM. This agent is essential for managing extensions and must be in a "Ready" state to function correctly. You can check its status in the Azure portal or by looking at the logs located at C:\WindowsAzure\Logs\WaAppAgent.log on your VM1 4.

Review Extension Logs

Inspect Extension Logs: Check the logs specific to the Splunk extension, which can typically be found in:

C:\WindowsAzure\Logs\Plugins\<ExtensionName>\<Version>\CommandExecution.log

C:\Packages\Plugins\<ExtensionName>\<Version>\Status\status.json

These logs will provide detailed information about what went wrong during the installation process1 4.

Common Issues and Solutions

Existing Failed Extensions: If there are any existing extensions with a failed provisioning state, new extensions may fail to install. Remove any failed extensions using PowerShell or the Azure portal before attempting to redeploy

Timeouts and Resource Constraints: Extensions have a limited time to run (usually 20 minutes). If your VM is low on resources or if other processes are consuming significant resources, it may lead to timeouts. Ensure your VM has adequate resources available during installation1 4.

Network and Dependency Requirements: Some extensions require network access to download packages or communicate with external services. Ensure that your VM can access necessary endpoints, such as Azure Storage or package repositories

Rerun the Extension

Remove and Redeploy: If all else fails, consider removing the Splunk extension and redeploying it. Use PowerShell commands like:
powershell Remove-AzVMExtension -ResourceGroupName "<ResourceGroup>" -VMName "<VMName>" -Name "<ExtensionName>"
After removal, you can redeploy the extension using your ARM template Please let us know if anything required!

Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-26T17:55:58.1+00:00

Hi rajesh.john, We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion. Thank You.!

Pavan Minukuri 765 Reputation points Microsoft Vendor

2024-11-27T20:08:19.6133333+00:00

Hi rajesh.john, We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion. Thank You.!
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Azure Policy to deploy VM application is success when run on our Resource group, but app is not installed

2 answers

Your answer