Network and Security Requirement for Azure Data Factory - Azure Integration Runtime

Question

Network and Security Requirement for Azure Data Factory - Azure Integration Runtime

Akash Verma 31

Data Source is SaaS with public endpoint. I am planning to do the movement of data from SaaS to SQL Server in Azure. I am using Azure Integration Runtime on ADF. I want to know below questions:

Saas --> SQL server

Network requirement from Saas side and Azure side ( Any IP whitelisting or port etc..)?
How to do these movement in a secured way ?
What is the flow of it. Connection initiates from ADF side or not ?
Can i do this in opposite way. (SQL Server to Saas)

Please suggest.

1 answer

Your answer

Answer 1

MartinJaffer-MSFT 26,236

Hello @Akash Verma and thank you for your question.
Are you using Azure SQL server, Azure SQL server Managed Instance, or a SQL server hosted on a VM in Azure?

3: ADF Initiates the connection, not the SaaS

4: Whether the SaaS is supported as a sink depends upon the SaaS and how you connect to it. Some SaaS have built-in connectors (see source-sink matrix here). Some you connect to via REST/Http datasets (not available as sink).

1: If you copy data by using the Azure integration runtime, configure a server-level firewall rule so that Azure services can access the server. Port 1433 is required for Azure SQL Database.

2: There are several security features available. Transparent Data Encryption may already be enabled on your SQL server.

Akash Verma 31 Reputation points

2020-09-03T22:54:21.423+00:00

Thanks @MartinJaffer-MSFT for your response. I am using the Azure Sql Server and Blob.

From your response on 1st part regarding networking requirement. 1433 Port is required for data sink(destination). What are the networking requirement for Data Source connection on azure side and Data source side ?

To add, what can i do to make this connection secure ?
MartinJaffer-MSFT 26,236 Reputation points

2020-09-04T00:18:33.207+00:00

SaaS------>ADF-------->SQL

For the stretch between SaaS and ADF, ensure the SaaS endpoint uses Https or TLS. This should encrypt that section of traffic. You already stated that your SaaS has a public endpoint, so I don't anticipate any access issues there.

All connections to Azure SQL Database require encryption (SSL/TLS) while data is in transit to and from the database. When you're authoring a pipeline by using JSON, add the encryption property and set it to true in the connection string. For Azure Storage, you can use HTTPS in the connection string.

In the SQL server firewall settings there is an option "Allow Azure services and resources to access this server" . Setting this to Yes will help whitelist the IP ranges used by Data Factory's Azure Integration Runtime.
If you prefer to do the whitelisting yourself, you can find ranges here.
Akash Verma 31 Reputation points

2020-09-07T14:14:49+00:00

Thanks for your response.

In this communication, even though connection initiates from ADF side via Azure IR, can we put firewall in Azure side for this communication ? if yes, how it will work then
MartinJaffer-MSFT 26,236 Reputation points

2020-09-08T20:14:26.42+00:00

If I understand you correctly, you want to put a firewall around the Data Factory, not the SQL server?

Well, there are a couple new features, Managed Virtual Network, and Private Endpoints, but I'm not confident you would still be able to access your SaaS when using these.

You can think of Data Factory as being separated into two pieces. One is the authoring tool, pipeline and service definitions, and triggers. The other being the compute (Integration Runtime) upon which your pipelines are run.
MartinJaffer-MSFT 26,236 Reputation points

2020-09-08T20:30:41.273+00:00

After looking a little more I decided to create an Azure IR with Vnet configuration and try it out.

As a test I pointed a web activity at google.com as that is definitely outside Azure. The debug run succeeded as you can see. However, as the vnet is managed, I can't do much to it.

My apologies if I have not been much help. I think you should be fine without placing a firewall between ADF and your SaaS. All data movement is initiated by Data Factory (with exception of blob event trigger, arguably). It is not like someone can push data to the Factory.
MartinJaffer-MSFT 26,236 Reputation points

2020-09-10T18:38:12.883+00:00

@Akash Verma I will now convert this conversation from comment to answer. If I answered your questions satisfactorily, please mark it as an accepted answer. This will help the community find what they are looking for.

Share via

Network and Security Requirement for Azure Data Factory - Azure Integration Runtime

1 answer

Your answer