Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,261 questions
Content
1 answer
Data Factory: turn on data flow debug error
Hello! I need the "Data Preview" for checking, and it require me to turn on "Data flow debug". However, error received and no ErrorCode. I can't do anything. Data flow debug Failed to setup debug session. Error: An error…
0 answers
Long running Databricks notebook but cumulative processing time is short
Hi Microsoft experts, I have a long running Databricks notebook tonight, and I can't see a reason why. My Data Factory pipeline which calls the notebook has been running reliably everyday for about 8 months, but this week it has had some problems and now…
asked 2023-01-13T09:29:24.8366667+00:00
James Hooi
0
Reputation points
commented 2023-01-29T03:12:39.82+00:00
James Hooi
0
Reputation points
0 answers
How the pass a session ID for a SOAP API in ADF
Hi all, Struggling for a while now with creating an API request in ADF to load data from a SOAP API. I'm having difficulties with passing a session ID to another API request which will provide the transactional data. step 1: API request (OpenSession)…
asked 2023-01-28T22:42:24.6466667+00:00
Remon van Ramele
0
Reputation points
asked 2023-01-28T22:42:24.6466667+00:00
Remon van Ramele
0
Reputation points
1 answer
Merging 2 Array variables
Hello All. I might be missing some simple aspect here but wanted to confirm : Is there any way in ADF to merge 2 Array variables without the use of any iteration activity. Like ask is as below: Input1= [1,2,3] Input2=[1,4,5] Output=[1,2,3,1,4,5] Note:…
asked 2023-01-28T07:35:35.7433333+00:00
Nandan Hegde
21,606
Reputation points
answered 2023-01-28T20:38:46.8233333+00:00
Dillon Silzer
33,176
Reputation points
0 answers
change column name in delta parquet file
hello, data fatory data flow is being used to upsert into delta parquet files. The delta parquet files have several columns. How is it possible to change the name of a column without affecting the data using the delta sink in the data flow? for…
1 answer
Your account or password is incorrect. If you don't remember your password, reset it now.
Hello! When I try to enter incorrect password, I give this error: Your account or password is incorrect. If you don't remember your password, reset it now.also when I try to click on reset now, I give this message: Verify your identity How would you like…
asked 2023-01-28T15:58:43.3833333+00:00
Anthony Hello
0
Reputation points
answered 2023-01-28T16:20:52.7966667+00:00
Dave Patrick
328.6K
Reputation points Microsoft MVP
0 answers
Change Data Capture in Azure Data factory High level resources does not ask for DelimitedText value, instead it takes comma(,) by default
Hi I am trying to implement CDC High Level Resources through Blob to SQL DB. But my mapping was incorrect as my source is delimited by "|", due to which I am unable to implement. Can we look into this? on how to change default comma(,) to…
asked 2023-01-26T23:04:18.0766667+00:00
Krishnamohan Nadimpalli
386
Reputation points
commented 2023-01-28T13:14:38.62+00:00
Krishnamohan Nadimpalli
386
Reputation points
0 answers
Can we convert R script to M code (Power Query)
As we are having few scripts in R which we are consuming in another scope, as per the new scope that needs be migrated to M (Power Query). If so, are there any best practices to migrate existing scripts to M code. Do we have any tools that can be used…
asked 2023-01-27T12:38:18.8+00:00
Manohar Kondragunta
0
Reputation points
edited the question 2023-01-28T04:47:39+00:00
AjayBathini-MSFT
0
Reputation points Microsoft Employee
0 answers
ADF copy data to cosmosdb producing errors
I created a copydata pipline in azure data factory to move data from a blob to cosmos mongo api database. All necessary dataset connections connected successfully .But while i debug or execute i get the errors below relating to the sink. Is there a way…
asked 2023-01-23T02:01:56.39+00:00
Shittu Adesoji (NHQ-AC-CONSULTANT)
0
Reputation points
commented 2023-01-28T04:17:39.3433333+00:00
BhargavaGunnam-MSFT
7,311
Reputation points Microsoft Employee
0 answers
pivot hour to minutes workflow
I need to pivot each row in a table as follows. From: To: Each hour of the day represented from h1 to h24 listing total minutes worked minus break. What would be the best workflow for this process?
asked 2023-01-21T17:49:58.8666667+00:00
Petar Novakovic
21
Reputation points
edited the question 2023-01-27T22:49:17.8866667+00:00
Petar Novakovic
21
Reputation points
1 answer
Where is the storage account key for cosmos db
I have a copy data pipeline which loads data from a json file within a blob to a cosmos DB collection. Now the sink is the cosmos dB connection . At creation time account selection method is Azure subscription , subscription is selected and then Azure…
asked 2023-01-27T14:46:21.0766667+00:00
Shittu Adesoji (NHQ-AC-CONSULTANT)
0
Reputation points
answered 2023-01-27T22:35:58.55+00:00
Dillon Silzer
33,176
Reputation points
0 answers
Airtable API Patch Request Fails in ADF
I'm trying to update data in Airtable using Rest API in ADF. I get an error. I'm trying to execute a PATCH. Here's my mapping. Example request from Airtable API documentation: Why is this not working? Error Message: Response:…
asked 2023-01-27T22:23:38.65+00:00
Jon Carlisle
66
Reputation points
asked 2023-01-27T22:23:38.65+00:00
Jon Carlisle
66
Reputation points
0 answers
Upsert to Azure SQL Database Failed because Interim Table Exists
I have created a Copy Activity on Azure Data Factory that moves the data from a CSV file hosted on Data Lake to Azure SQL Database. When I select the write behavior to be INSERT, the pipeline runs fine. But when I changed to UPSERT and provide the proper…
1 answer
One of the answers was accepted by the question author.
How to update data into Azure storage table from datafactory?
1) Can i use storage table for capture log details of pipeline via datafactory. will it allow insert new values in table? 2) I want to update specific column in storage table. how do i update the column which activity I should use? like I want to…
asked 2020-10-13T12:07:05.143+00:00
THIMMAIAH GARI,PRASHANTH,,
191
Reputation points
commented 2023-01-27T17:43:38.41+00:00
Mehdi Nikkhah
0
Reputation points
0 answers
Error to set lookup field in N:1 relationship while writing data to Dynamics CRM from Azure Data Factory
While trying to sync data from sql staging table to Account entity in CRM, I faced an issue in populating the multi-lookup column. The input column is Manager Name that is of string data type. The destination field is dfws_sareportingmanager (GUID data…
asked 2023-01-25T21:03:04.89+00:00
AnaC
0
Reputation points
commented 2023-01-27T17:40:38.6833333+00:00
Pratik Somaiya
4,026
Reputation points
1 answer
Azure Storage explorer stuck on Loading.. any suggestion
i am facing this problem on my intel arc gaming pc running on windows 11 rightnow. when i Try to connect to blob storage in Azure Storage Emulator its stuck on Loading.. Was working last week before I did a windows update and storage explorer update. I…
asked 2023-01-27T08:07:38.8933333+00:00
AARON JEFFERY
0
Reputation points
answered 2023-01-27T16:02:55.7933333+00:00
Marshaljs
26,191
Reputation points
0 answers
Problem with ADF Dataflow pipeline when writing data to SQL Server
I have an ADF Dataflow pipeline that reads data from a data lake and writes it to a SQL Server table. I have a problematic column called "gdtxft" that I converted from binary to string using the 'toString' function in the Derived column…
asked 2023-01-25T10:06:58.0633333+00:00
Smagulov, Alisher [Norsec Delta Projects LLP]
0
Reputation points
commented 2023-01-27T15:55:32.6633333+00:00
Subashri Vasudevan
4,296
Reputation points
1 answer
Does Interactive authoring of the integration runtime have to be activated manually?
Hello! I just wanted to know if there's a way to automate the enabling of the interactive authoring of the AutoResolveIntegrationRuntime. Thanks!
asked 2022-06-29T16:07:00.26+00:00
Federico Arturo Ramirez Seidel
26
Reputation points
commented 2023-01-27T15:53:05.7033333+00:00
Marc Scirri
1
Reputation point
1 answer
How to convert data row into valid json in ADF Dataflow
I have data in tabular format. As part of dataflow transformation want add another derived column like below. Basically converting column data into valid Json string
asked 2023-01-27T10:48:56.6433333+00:00
Vivek, Anand
0
Reputation points
edited an answer 2023-01-27T14:47:38.1833333+00:00
AnnuKumari-MSFT
19,116
Reputation points Microsoft Employee
0 answers
Very Old System: Failed to load SOAP Service definition file.
defunct quetion, posted it before editing, will do another to fix it
asked 2023-01-25T13:12:45.45+00:00
Bing sling
0
Reputation points
commented 2023-01-27T14:41:01.3633333+00:00
BhargavaGunnam-MSFT
7,311
Reputation points Microsoft Employee