I want to access aws cloudwatch logs from azure datafactory and process them using a python script

Question

1. Can I use authly instead of access token and secret keys to access cloud watch logs?

Answer 1

@Lenin - Thanks for the question and using MS Q&A platform.

To access AWS CloudWatch logs from Azure Data Factory, you can use the via REST API.

For more details, refer to MS Q&A thread addressing similar issue: https://learn.microsoft.com/en-us/answers/questions/962565/want-to-access-aws-cloudwatch-from-adf

Regarding your question, I'm not sure what you mean by "authly". If you are referring to AWS Cognito authentication, it is not supported in Azure Data Factory.

The following properties are supported for the REST linked service:

For more details, refer to Copy and transform data from and to a REST endpoint by using Azure Data Factory.

How to access aws cloudwatch logs in Azure?

If you’re working in Amazon Web Services (AWS) or have a multi-cloud environment that includes AWS, a solid logging and monitoring solution should be an essential part of your deployment and monitoring strategy. In AWS, this takes the form of CloudWatch, a service designed to collect diagnostic logging, utilization metrics, and events inside your service(s). However, if you’re looking for a fast, scalable solution to query your logs, it is possible to take the data collected in your CloudWatch Log Groups and send that information over to Azure Data Explorer and then use the power of Kusto to provide powerful analysis and reporting over any volume of your logging data for all your log groups.

For more details, refer to Exporting AWS CloudWatch Logs to Azure Data Explorer

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.