Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,950 questions
Incremental ETL From Mysql to Azure
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 49%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EN%3C/text%3E%3C/svg%3E)
NOC_F
0
Reputation points
Hello Professionals,
I am having a totally new experience with AZURE as I have never worked on it , but due to some conditions I have to use AZURE ,
I am having some issues In making a incremental Data Flow that should extract data from 4 MySQL Server to Azure and then to Power BI Desktop, I am tring multiple things making pipeline in Data Factory starting with Copy Data activity have already made Azure SQL Database and all , but nothing is working, 2 out of 5 Time I get connection Error from Azure Sql Database that I made and the MySql User I have is Read only.
Please provide me your assistance
Thank You
{count} votes
-
Vinodh247-1375 12,426 Reputation points2024-05-27T09:30:54.5366667+00:00 2 out of 5 Time I get connection Error from Azure Sql Database that I made and the MySql User I have is Read only.
are you trying to extract data from mysql or azure sql
-
NOC_F 0 Reputation points
2024-05-27T09:47:41.29+00:00 Yes , Is there any option we could just make the data flow to Power BI Desktop without storing anywere, what Steps should I take to make this flow
-
Vinodh247-1375 12,426 Reputation points
2024-05-27T10:15:15.6433333+00:00 -
Vinodh247-1375 12,426 Reputation points
2024-05-27T10:16:32.17+00:00 okay, my understanding is that you are trying to pull data from 4 mysql databases? now your issue is connecting to source itself or adding up the incremental data?
-
NOC_F 0 Reputation points
2024-05-27T11:07:00.1166667+00:00 I have tried the above mathod but the data is to big and i can not connect through Directquery also as I have to manipulate tables, Table size ranges between 25GB - 45GB
-
Vinodh247-1375 12,426 Reputation points
2024-05-27T14:19:05.1066667+00:00 did you try using the parameters?
https://www.phdata.io/blog/loading-large-datasets-power-bi-using-parameters/
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu 11,775 Reputation points Microsoft Vendor2024-05-28T06:25:48.6466667+00:00 @NOC_F We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Sign in to comment