How can I connect Data Factory to AmazonMQ Queue?

Question

I am looking to create a pipeline that would pull data from amazonmq queue and place it in sql server.

I do not see a linked service connector for amazonmq.

Would I need to use http connector?

Some other method?

I can't be the only person that wants to do this. Does anyone have an example of this?

Answer 1

Hi @Blasko, Dan W
Thank you for your question and using the Microsoft Q&A platform.
While Azure Data Factory does not have a built-in connector for Amazon MQ, you can still achieve this by using a combination of different services and components.

1. Using HTTP Connector

You can use the HTTP connector in Azure Data Factory to pull messages from Amazon MQ. Set up an HTTP endpoint to read messages from Amazon MQ. Create a pipeline in Azure Data Factory that uses the HTTP connector to call this endpoint and retrieve messages. Process the messages and use a Copy Activity to insert the data into SQL Server.

2. Using AWS Lambda and Azure Functions

Another approach is to use AWS Lambda to read messages from Amazon MQ and then send them to an Azure Function, which can then insert the data into SQL Server. Create an AWS Lambda function to read messages from Amazon MQ. Invoke an Azure Function from the Lambda function, passing the message data. Azure Function processes the data and inserts it into SQL Server.

For more help please refer to the below link:
https://learn.microsoft.com/en-us/answers/questions/1526874/integration-with-aws-mq

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.