Breyta

Deila með

Upload logs to Azure Monitor

  • Grein

Periodically, you can export logs and then upload them to Azure. Exporting and uploading logs also creates and updates the data controller, SQL managed instance, and PostgreSQL server resources in Azure.

Before you begin

Before you can upload logs, you need to:

  1. Create a log analytics workspace
  2. Assign ID and shared key to environment variables

Examples in this article use angle brackets < ... > to identify values that you need to replace before you run the script. Replace the brackets and the values inside the brackets.

Create a log analytics workspace

To create a log analytics workspace, execute these commands to create a Log Analytics Workspace and set the access information into environment variables.

Note

Skip this step if you already have a workspace.

az monitor log-analytics workspace create --resource-group <resource group name> --workspace-name <some name you choose>

Example output:

{
  "customerId": "00000000-0000-0000-0000-000000000000",
  "eTag": null,
  "id": "/subscriptions/<Subscription ID>/resourcegroups/user-arc-demo/providers/microsoft.operationalinsights/workspaces/user-logworkspace",
  "location": "eastus",
  "name": "user-logworkspace",
  "portalUrl": null,
  "provisioningState": "Succeeded",
  "resourceGroup": "user-arc-demo",
  "retentionInDays": 30,
  "sku": {
    "lastSkuUpdate": "Thu, 30 Jul 2020 22:37:53 GMT",
    "maxCapacityReservationLevel": 3000,
    "name": "pergb2018"
  },
  "source": "Azure",
  "tags": null,
  "type": "Microsoft.OperationalInsights/workspaces"
}

Assign ID and shared key to environment variables

Save the log workspace analytics customerId as an environment variable to be used later:

SET WORKSPACE_ID=<customerId>

This command returns the access keys required to connect to your log analytics workspace:

az monitor log-analytics workspace get-shared-keys --resource-group MyResourceGroup --workspace-name MyLogsWorkpace

Example output:

{
  "primarySharedKey": "<primarySharedKey>==",
  "secondarySharedKey": "<secondarySharedKey>=="
}

Save the primary key in an environment variable to be used later:

SET WORKSPACE_SHARED_KEY=<primarySharedKey>

Verify environment variables

Check to make sure that all environment variables required are set if you want:

echo %WORKSPACE_ID%
echo %WORKSPACE_SHARED_KEY%

With the environment variables set, you can upload logs to the log workspace.

Configure automatic upload of logs to Azure Log Analytics Workspace in direct mode using az CLI

In the direct connected mode, Logs upload can only be set up in automatic mode. This automatic upload of metrics can be set up either during deployment or post deployment of Azure Arc data controller.

Enable automatic upload of logs to Azure Log Analytics Workspace

If the automatic upload of logs was disabled during Azure Arc data controller deployment, run the below command to enable automatic upload of logs.

az arcdata dc update --name <name of datacontroller> --resource-group <resource group> --auto-upload-logs true
#Example
az arcdata dc update --name arcdc --resource-group <myresourcegroup> --auto-upload-logs true

Enable automatic upload of logs to Azure Log Analytics Workspace

If the automatic upload of logs was enabled during Azure Arc data controller deployment, run the below command to disable automatic upload of logs.

az arcdata dc update --name <name of datacontroller> --resource-group <resource group> --auto-upload-logs false
#Example
az arcdata dc update --name arcdc --resource-group <myresourcegroup> --auto-upload-logs false

Configure automatic upload of logs to Azure Log Analytics Workspace in direct mode using kubectl CLI

Enable automatic upload of logs to Azure Log Analytics Workspace

To configure automatic upload of logs using kubectl:

  • ensure the Log Analytics Workspace is created as described in the earlier section
  • create a Kubernetes secret for the Log Analytics workspace using the WorkspaceID and SharedAccessKey as follows:
apiVersion: v1
data:
  primaryKey: <base64 encoding of Azure Log Analytics workspace primary key>
  workspaceId: <base64 encoding of Azure Log Analytics workspace Id>
kind: Secret
metadata:
  name: log-workspace-secret
  namespace: <your datacontroller namespace>
type: Opaque

  • To create the secret, run:

    kubectl apply -f <myLogAnalyticssecret.yaml> --namespace <mynamespace>

  • To open the settings as a yaml file in the default editor, run:

    kubectl edit datacontroller <DC name> --name <namespace>

  • update the autoUploadLogs property to "true", and save the file

Enable automatic upload of logs to Azure Log Analytics Workspace

To disable automatic upload of logs, run:

kubectl edit datacontroller <DC name> --name <namespace>
  • update the autoUploadLogs property to "false", and save the file

Upload logs to Azure Monitor in indirect mode

To upload logs for SQL Managed Instance enabled by Azure Arc and Azure Arc-enabled PostgreSQL servers run the following CLI commands-

  1. Export all logs to the specified file:

    Note

    Exporting usage/billing information, metrics, and logs using the command az arcdata dc export requires bypassing SSL verification for now. You will be prompted to bypass SSL verification or you can set the AZDATA_VERIFY_SSL=no environment variable to avoid prompting. There is no way to configure an SSL certificate for the data controller export API currently.

    az arcdata dc export --type logs --path logs.json  --k8s-namespace arc

  2. Upload logs to an Azure monitor log analytics workspace:

    az arcdata dc upload --path logs.json

View your logs in Azure portal

Once your logs are uploaded, you should be able to query them using the log query explorer as follows:

  1. Open the Azure portal and then search for your workspace by name in the search bar at the top and then select it.
  2. Select Logs in the left panel.
  3. Select Get Started (or select the links on the Getting Started page to learn more about Log Analytics if you are new to it).
  4. Follow the tutorial to learn more about Log Analytics if this is your first time using Log Analytics.
  5. Expand Custom Logs at the bottom of the list of tables and you will see a table called 'sql_instance_logs_CL' or 'postgresInstances_postgresql_logs_CL'.
  6. Select the 'eye' icon next to the table name.
  7. Select the 'View in query editor' button.
  8. You'll now have a query in the query editor that will show the most recent 10 events in the log.
  9. From here, you can experiment with querying the logs using the query editor, set alerts, etc.

Automating uploads (optional)

If you want to upload metrics and logs on a scheduled basis, you can create a script and run it on a timer every few minutes. Below is an example of automating the uploads using a Linux shell script.

In your favorite text/code editor, add the following script to the file and save as a script executable file - such as .sh (Linux/Mac), .cmd, .bat, or .ps1 (Windows).

az arcdata dc export --type logs --path logs.json --force --k8s-namespace arc
az arcdata dc upload --path logs.json

Make the script file executable

chmod +x myuploadscript.sh

Run the script every 20 minutes:

watch -n 1200 ./myuploadscript.sh

You could also use a job scheduler like cron or Windows Task Scheduler or an orchestrator like Ansible, Puppet, or Chef.

Related content

Upload metrics, and logs to Azure Monitor

Upload usage data, metrics, and logs to Azure Monitor

Upload billing data to Azure and view it in the Azure portal

View Azure Arc data controller resource in Azure portal