JDBC Query as Source in Azure Data Factory Pipeline

Monica Deutsch 1

Hi,

I need to query a JDBC connection as the source of my Azure Data Factory Pipeline (the data source is a third party system which only offers access via JDBC).
So is there any possibility to do that with ADF?
If not, which alternatives do I have in Azure to query the JDBC connection and write the data into my Azure SQL DB?

Thanks for your help!

PRADEEPCHEEKATLA 90,246 Reputation points

2020-09-08T07:54:14.273+00:00

Hello anonymous user,

Welcome to Microsoft Q&A platform.

Could you please share more details on the third-party data source which want to via JDBC Query as source in Azure Data Factory pipeline?
Monica Deutsch 1 Reputation point

2020-09-08T14:58:39.46+00:00

Unfortunately, I don't have much further information at this point.
My customer wants to migrate their on premises ERP database to an SaaS solution their ERP provider offers. They only allow JDBC access to their database (in AWS).
I also don't have any testing connection from them, that I could play arround with (I was planning on using the JDBC connection of my own Azure SQL DB for testing).

I am just looking for the simplest (and most cost effective) way in Azure to query data from a JDBC connection and write it into staging tables in my Azure SQL DB, without any transformation. I just want to execute a select statement via JDBC and write the resulting data into an Azure SQL DB. Is there any straight forward possibility to do this?

The only ways I found so far (and am testing now) are using a custom .NET app running as a web job or using Databricks and a Python/Scala notebook, which is not an ideal solution because we don't need Databricks for anything else.
Kevin Franck 1 Reputation point

2022-12-09T00:00:00.767+00:00

Realize this is rather old post but did you ever find a solution to using ADF + Integration Runtime + JDBC Connector?

1 answer

PRADEEPCHEEKATLA 90,246 Reputation points

2020-09-09T08:13:34.53+00:00

Hello anonymous user,

Unfortunately, there is no specific connector available to get data via JDBC query as source in Azure Data Factory pipeline.

I would suggest you to provide feedback on the same:

https://feedback.azure.com/forums/270578-data-factory

All of the feedback you share in these forums will be monitored and reviewed by the Microsoft engineering teams responsible for building Azure.

Hope this helps. Do let us know if you any further queries.

----------------------------------------------------------------------------------------

Do click on "Accept Answer" and Upvote on the post that helps you, this can be beneficial to other community members.
Please sign in to rate this answer.
Monica Deutsch 1 Reputation point

2020-09-09T12:15:02.477+00:00

Thank you for your reply, I will post my idea in the forum you linked.

And thank you for confirming what I already suspected, that it is not possible to query JDBC as a datasource in ADF.
Do you know of any other Azure Service that could query JDBC and write the result into an Azure SQL DB table?

PRADEEPCHEEKATLA 90,246 Reputation points

2020-09-11T05:51:11.687+00:00

Hello anonymous user,

Unfortunately, without knowing the details on the third-party data source which want to via JDBC Query as source, it hard to provide the solution or any recommendations.

Vaibhav Sharma 1 Reputation point

2020-11-30T09:59:15.5+00:00

Hi Pradeep, Could you please share some details on how did you manage to get the Cloud ERP data to Azure SQL DB ? Keen to know as my requirements are pretty Similar.

Cheers
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

JDBC Query as Source in Azure Data Factory Pipeline

1 answer

Your answer