Public REST API connection usinga Self-Hosted Integration Runtime hosted on a VM within my own Azure Virtual Network

Question

I am looking to connect to a public REST API to return a payload of data. Can a Self-Hosted Integration Runtime hosted on a VM within my own Azure Virtual Network be configured to the public REST API so data can be ingested in to Azure Blob storage?

I would expect the Self-Hosted Integration Runtime hosted on a VM within my own Azure Virtual Network to have a security perimeter (firewall, rules configured) before the data is entered into ADF and any data movement within ADF should be over a Private Endpoint connection. Is this architecture possible?

Thanks.

Answer 1

@Patel, Mehal Apologies for the delay response!

Yes, it is possible to use a Self-Hosted Integration Runtime hosted on a VM within your own Azure Virtual Network to connect to a public REST API and ingest data into Azure Blob storage.

To do this, you would need to configure the Self-Hosted Integration Runtime to allow outbound traffic to the public REST API. You can do this by configuring the firewall rules on the VM hosting the Self-Hosted Integration Runtime to allow outbound traffic to the public REST API.

Once the Self-Hosted Integration Runtime is configured to allow outbound traffic to the public REST API, you can use Azure Data Factory to ingest the data into Azure Blob storage. You can configure the data movement within Azure Data Factory to use a Private Endpoint connection to ensure that the data is transferred securely over a private network connection.

It's important to note that when you connect to a public REST API, you should ensure that you are following best practices for securing your connection and protecting your data. This may include using SSL/TLS encryption, verifying the identity of the API endpoint, and using secure authentication methods.

I hope this helps! Let me know if you have any other questions.