Hi, I have to build a Python ETL pipeline to update a table that is used for a PowerBI report. But I have 2 problems that I havent been able to solve: One of the ETL inputs is a self-hosted on-prem SQL server database. I can connect to this SQL DB within Azure Data Factory using a Linked Service that uses a specific Integration Runtime (not AutoResolveIntegrationRuntime). But I cannot connect to this DB using Azure Functions pyodbc function (using Python). My infra team doesnt have a solution to this. Does anyone know how to connect this DB within Azure functions? On a side note, I can connect to other SQL DB using AutoResolveIntegrationRuntime in Linked Service & these DB connect within Azure Functions just fine. Secondly, if connection to this on-prem DB cannot be possible within Azure function, whats an alternative to load the data from this SQL DB using ADF activities & give that as input to Azure Function Activity so that my python code can process it? Thanks in advance for your answers.

A community member has associated this post with a similar question:
How to setup an ETL pipeline using DB that cannot be accessed via Azure Functions

Only moderators can edit this content.

How to setup an ETL pipeline to update a DB table using Azure Functions and ADF & copy data

Deepak 0

Hi,

I have to build a Python ETL pipeline to update a table that is used for a PowerBI report.

But I have 2 problems that I havent been able to solve:

One of the ETL inputs is a self-hosted on-prem SQL server database. I can connect to this SQL DB within Azure Data Factory using a Linked Service that uses a specific Integration Runtime (not AutoResolveIntegrationRuntime). But I cannot connect to this DB using Azure Functions pyodbc function (using Python). My infra team doesnt have a solution to this. Does anyone know how to connect this DB within Azure functions? On a side note, I can connect to other SQL DB using AutoResolveIntegrationRuntime in Linked Service & these DB connect within Azure Functions just fine.
Secondly, if connection to this on-prem DB cannot be possible within Azure function, whats an alternative to load the data from this SQL DB using ADF activities & give that as input to Azure Function Activity so that my python code can process it?

Thanks in advance for your answers.