Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
1,089 questions
0 answers
How to improve Copy Data runtime on Azure Data Factory
Hi, I'm copying data from Amazon S3 to Azure Data Lake Storage Gen2. The source contains mutliple parquet files with same schema and the target is a single parquet file that should contain the merged inforamtion from my source, so I set the copy…
Azure
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,196 questions
asked 2024-07-26T21:03:14.7266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(25.6, 50%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVZ%3C/text%3E%3C/svg%3E)
Violet Zeng
30
Reputation points
asked 2024-07-26T21:03:14.7266667+00:00
Violet Zeng
30
Reputation points
0 answers
How to pass linked service parameter to Data Flow sink for error logging?
How do I pass in a parameter to the error logging linked service in a data flow? Here is my parameterized linked service: In my data flow sink I am trying to write skipped rows to blob storage: When I click on Debug Settings in the data flow editor I…
asked 2024-07-25T18:34:29.52+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(160, 93%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJB%3C/text%3E%3C/svg%3E)
Jenkins, Ben
0
Reputation points
commented 2024-07-26T20:57:22.1233333+00:00
Jenkins, Ben
0
Reputation points
1 answer
Run R script on ADF
i have a very small R script which needs to be run in ADF. What is the ideal solution for this ? Azure batch /Azure Function /Azure automation account /vm's(VM is existing and not new, how to connect vm's from ADF and execute R file). I got to know R…
asked 2022-08-29T13:53:06.907+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 67%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
Prakash14
121
Reputation points
commented 2024-07-26T20:55:51.41+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 57.00000000000001%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEZ%3C/text%3E%3C/svg%3E)
Eric Zhang
5
Reputation points
1 answer
Error "Cannot read properties of undefined (reading '0')"
I'm using Azure Data Factory online via a browser. I'm fairly new to ADF and have yet to complete any ADF tutorial due to various problems I've encountered. I'm doing a tutorial on ADF from Udemy. I'm running a pipeline to copy data from a…
asked 2024-07-26T10:25:18.1033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 77%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECR%3C/text%3E%3C/svg%3E)
Chuck Roberts
65
Reputation points
edited an answer 2024-07-26T20:35:50.9633333+00:00
Amira Bedhiafi
20,176
Reputation points
1 answer
Copy Data Lineage Activity is not reported in Purview
Hi, Context: I have a data pipeline in ADF. The pipeline contains the following activities A total of 15 number of copy data activity Total 1 number of Data Flow activity The data source for all activities is a common blob storage account. The data…
asked 2024-07-26T14:58:02.5+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 26%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
Ravineesh
40
Reputation points
answered 2024-07-26T20:30:58.52+00:00
Amira Bedhiafi
20,176
Reputation points
1 answer
Will the Azure Synapse/Data Factory Salesforce (Legacy) connector still work after the deprecation date?
Hello, I'm hoping to get additional details about the deprecation of the Azure Synapse/Data Factory Salesforce (Legacy) connector. The article linked below does not specify whether it will become unusable or just not be supported. Will it continue to…
asked 2024-07-26T18:42:02.56+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 28.999999999999996%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
Richard
0
Reputation points
answered 2024-07-26T18:56:20.13+00:00
hossein jalilian
5,560
Reputation points
1 answer
How to modify the order number of test cases in Azure Test Plan
I have uploaded a CSV file containing a test design to Azure DevOps. The file includes 43 test cases with order numbers ranging from 2 to 44. I want to change the starting order number to 1. However, I am having difficulty editing the order number for…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,196 questions
Office Development
Office Development
Office: A suite of Microsoft productivity software that supports common business tasks, including word processing, email, presentations, and data management and analysis.Development: The process of researching, productizing, and refining new or existing technologies.
3,720 questions
Microsoft Entra ID
Microsoft Entra ID
A Microsoft Entra identity service that provides identity management and access control capabilities. Replaces Azure Active Directory.
20,629 questions
asked 2024-07-26T18:19:39.1733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 85%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHR%3C/text%3E%3C/svg%3E)
HEMARITHI R
0
Reputation points
answered 2024-07-26T18:48:25.3133333+00:00
hossein jalilian
5,560
Reputation points
2 answers
Azure datalake creation methods
Hi friends, we need to create a datalake (ADLS) with bronze, silver, and gold layers. One of our colleagues suggested it is a better idea to create ADLS through programming, but we plan to create it manually. Are there any advantages of each approach …
asked 2024-07-24T10:24:14.9566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 15%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anshal
2,246
Reputation points
answered 2024-07-26T18:48:18.1633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 98%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
KarishmaTiwari-MSFT
19,032
Reputation points Microsoft Employee
1 answer
adf from configtable
Hi, how to load tablename from config table and using copy activity at front end... how it will execute it.... let say copy activtiy lookup for tablename then how copy activity will take that...please help with screenshot ...
asked 2024-07-21T20:01:20.1833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Vineet S
445
Reputation points
commented 2024-07-26T18:26:41.9433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 46%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EN%3C/text%3E%3C/svg%3E)
NIKHILANETHIKUNTA
155
Reputation points Microsoft Vendor
0 answers
Staging of Source Data in Parquete vs SQL DB for DWH with ADF for Delta-Loads
Hi there, I have some "conceptual" question I would like to discuss. New DWH should is beeing build on Azure using ADF and SQL for Dimensions/Facts. Today, we have also running the landing on SQL DB. Also we are only loading delta datasets…
asked 2024-07-23T06:58:39.9333333+00:00
Jörg Lang
120
Reputation points
commented 2024-07-26T18:16:49.0633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Bhargava-MSFT
29,266
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
How can I accept and process output form stored procedure in azure data dactory ?
I am trying to create a pipeline in the Azure data factory and want to process the output returned by my stored procedure. Till now I am able to successfully connect to my stored procedure in azure SQL database. This is the error I am getting while…
asked 2024-07-26T12:16:43.0233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 60%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPM%3C/text%3E%3C/svg%3E)
Palash Mathur
40
Reputation points
accepted 2024-07-26T17:54:55.04+00:00
Palash Mathur
40
Reputation points
0 answers
Azure Data Factory pipeline is failing as i have migrated to Azure MySQL Flexi server from mySQL single server
I am using azure Data factory which has many pipelines. Pipeline pulls data from Oracle on-Prim data base using integration runtime and puts the data into Azure MySQL database. Initially the destination was a MySQL single server, now i have migrated to…
asked 2024-07-25T13:31:44.5766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 42%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMK%3C/text%3E%3C/svg%3E)
Manish Kumar
0
Reputation points
commented 2024-07-26T17:03:39.1+00:00
Bhargava-MSFT
29,266
Reputation points Microsoft Employee
0 answers
i can't activate Azure P2 license and it made me set up another tenant .What can i do for this situation
I created a new azure free tenant to study for my AZ-104 but unfortunately,I can't activate a P2 license and it mademe set up another tenant.
asked 2024-07-26T16:09:06.7266667+00:00
Sangeetha T S
60
Reputation points
commented 2024-07-26T16:49:16.86+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 1%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
Patchfox
3,811
Reputation points
1 answer
Connect Google drive to Azure without GCP and without creating a copy
Hello I was just wondering if there's a way to use sort of virtualized data transfer instead of creating a copy of the data on Azure. As far as I understand all explained ways include creating a copy of the file and moving to Azure to index it. Is there…
asked 2024-07-25T08:03:41.71+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 43%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETM%3C/text%3E%3C/svg%3E)
Tony Minozi
20
Reputation points
commented 2024-07-26T16:27:15.26+00:00
Tony Minozi
20
Reputation points
4 answers
Xero connection error using oAuth2
I'm trying to establish the connection to Xero using the Xero connector via oAuth2 with the clientId and clientSecret. In the UI form, the required fields are: ClientID ClientSecret TenantID RefreshToken After filling all fields with the correct…
asked 2024-07-23T14:53:58.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 55.00000000000001%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EOM%3C/text%3E%3C/svg%3E)
Oleksandr Muzychenko
0
Reputation points
commented 2024-07-26T16:08:27.93+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
480
Reputation points Microsoft Vendor
3 answers
Column truncate issue - Copy file to DB ADF pipeline
Hi All, I have a pipeline where I am copying data from a file to a database. In the file, I have a few columns with extra spaces that I need to trim or truncate. I am not getting any option to use any transformation or function (dynamic content) on the…
asked 2024-07-19T17:52:29.9766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(96, 24%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
ADF_Coder
0
Reputation points
commented 2024-07-26T16:02:51.4633333+00:00
Chandra Boorla
480
Reputation points Microsoft Vendor
2 answers
ADF Copy Activity - Emply col issue
Hi Team, I have a pipeline where I am copying the data from the file to a database. The file had one extra empty column in between, and my copy activity failed with the below error: Message: Invalid Excel header with empty value It failed because the…
asked 2024-07-20T02:14:27.9266667+00:00
ADF_Coder
0
Reputation points
commented 2024-07-26T15:54:00.0666667+00:00
Chandra Boorla
480
Reputation points Microsoft Vendor
1 answer
ADF Copy Activity - To Remove records where all cols are Blank
Hi All, I have a copy activity that copies the data from an Excel file to a database table. It seems we are getting some blank rows in the source file for all the columns. In the database table, it is reflected as NULL. We need to restrict these rows…
1 answer
One of the answers was accepted by the question author.
How does ADF Linked Service for Azure Databricks aligns with Job Compute Policy defined inside Azure Databricks
This is a two-part Question. First Part Context: I have ADF which contains several data pipelines. Some pipeline also includes a databricks notebook as an activity. I have created Linked Service Azure Data Factory to facilitate the pipeline which…
0 answers
Azure DataFactory won't sink to table
Good day, Currently I have a datafactory that retrieves data from table X. This data will be checked and transformed (1 column) and then persisted back into that table. When I run this dataflow I get the result that 31 rows are written with some updated…
asked 2024-07-26T10:01:22.1033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 55.00000000000001%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBV%3C/text%3E%3C/svg%3E)
Bart Van Gestel
0
Reputation points
commented 2024-07-26T14:26:44.8366667+00:00
AnnuKumari-MSFT
32,161
Reputation points Microsoft Employee