Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,596 questions
Content
1 answer
insert record from azure datafactory into a delta table generated in databricks
Hi team, i'm trying to insert value into a delta table from datafactory, the table has been created in databricks and in the table properties i setted "delta.enableChangeDataFeed=true", for this reason the protocol minWriterVersion of my delta…
asked 2022-03-02T12:08:37.73+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 61%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAN%3C/text%3E%3C/svg%3E)
Antonio Nuzzo
11
Reputation points
commented 2023-09-24T02:03:35.55+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 99%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETF%3C/text%3E%3C/svg%3E)
Tim Frazer
0
Reputation points
1 answer
Generated ADF pipelibe using Python
Hello, I am working on the dynamic data pipeline framework were i can generate Azure data pipeline programatically. till now, i have done for creating copy activiyt and dataflow. the final step is, need to link between this two by creating another…
asked 2023-09-19T11:17:33.5466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 37%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECT%3C/text%3E%3C/svg%3E)
CON-Thirusenthilkumar Pandiyan
5
Reputation points
edited a comment 2023-09-23T12:04:43.12+00:00
CON-Thirusenthilkumar Pandiyan
5
Reputation points
2 answers
Adding Azure Data Factory Managed Identity to Azure SQL DB Failes - Error Msg 33132
When trying to grant a Data Factory instance access to an Azure SQL DB I get the following error. "Msg 33132, Level 16, State 1, Line 1 This principal type is not supported in Azure SQL Database." Using this command - CREATE USER…
asked 2023-02-15T02:06:09.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 34%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESC%3C/text%3E%3C/svg%3E)
Scott Cooke (Agile Analytics)
5
Reputation points
commented 2023-09-23T11:41:05.98+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 89%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDE%3C/text%3E%3C/svg%3E)
David EDWARDS
0
Reputation points
1 answer
how to fix -Your account is under review
Hi , I am facing below message while login Azure free Account. Your account is under review Kindly help met to get solved. Thanks in advance.
asked 2023-09-17T14:22:05.4766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 72%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
mkeshwar
0
Reputation points
commented 2023-09-22T19:26:51.3933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 79%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERR%3C/text%3E%3C/svg%3E)
Rahul Randive
4,291
Reputation points Microsoft Employee
2 answers
Stored procedure with parameters not working in azure data factory
In my Azure Data Factory pipeline, I have an "if" activity. Within the "if" activity, I have a "stored procedure" activity that executes a delete statement and requires a parameter. In the ADF "stored procedure"…
asked 2023-09-15T17:12:36.1233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 59%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDC%3C/text%3E%3C/svg%3E)
Deirdre Clark
0
Reputation points
commented 2023-09-22T18:24:02.87+00:00
Deirdre Clark
0
Reputation points
1 answer
Dataset works with pipeline but does not connect with dataflow
Hello, On azure data factory I created some datasets that work flawlessly on pipelines: The pipeline execution works as expected: And I get the data I need. But when I try to use that dataset in a dataflow I get a connection error: It says it could be…
asked 2022-05-30T07:22:16.02+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 60%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
Sergi Arbós Mauri
6
Reputation points
commented 2023-09-22T17:40:26.02+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 77%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMK%3C/text%3E%3C/svg%3E)
Manish Kumar
0
Reputation points
0 answers
ADF Ingestion
What different ways you can ingest the data using ADF and Synapse
asked 2023-09-20T08:42:05.09+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 15%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anshal
1,201
Reputation points
commented 2023-09-22T17:26:52.07+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
BhargavaGunnam-MSFT
19,316
Reputation points Microsoft Employee
1 answer
How to connect to Azure PostgreSQL with AAD login, from ADF or Synapse pipeline?
Hello, I have a PostgreSQL (PG) database that exclusively permits Azure Active Directory (AAD) logins. I want to establish a linked service in Azure Synapse to import data from this database. If I obtain an authentication token from a user who belongs…
asked 2023-09-22T14:57:51.6933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 85%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDN%3C/text%3E%3C/svg%3E)
Deba Nayak
41
Reputation points
edited a comment 2023-09-22T17:11:05.7066667+00:00
Amira Bedhiafi
4,071
Reputation points
1 answer
Data Flow using Temp File on SFTP by Default
I am creating a process that picks up a txt file based on "blob created" event trigger in ADF Data Flow. Once this txt file triggers, my data flow picks up the file translates and maps it to a CSV. On my sink I have it connected to an SFTP…
asked 2023-09-22T15:10:16.6066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 31%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESO%3C/text%3E%3C/svg%3E)
Sasha Oelsner
0
Reputation points
commented 2023-09-22T16:23:00.8866667+00:00
Amira Bedhiafi
4,071
Reputation points
1 answer
How to bring through text readable fields (for lookup/choice fields in dataverse) when copying data using the azure synapse link or azure data factory
I have two situations where I copy data from Dataverse into other storage locations: using azure synapse link to copy data from dataverse into azure data lake storage using azure data factory copy activity to copy data from dataverse into azure sql…
Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
559 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
3,536 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
7,983 questions
asked 2023-09-22T15:07:03.49+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(169.60000000000002, 84%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHC%3C/text%3E%3C/svg%3E)
Hannah Chant
20
Reputation points
answered 2023-09-22T16:11:08.49+00:00
Amira Bedhiafi
4,071
Reputation points
0 answers
Azure Data Factory set variable add backslash before double quote.
In Azure Data Factory, I need to copy the output of a REST API, which is an array of nested JSONs, to a CSV file. I used a loop for the array and dynamic content for the copy activity's mapping. I have a Pipelines_Mappings variable, it contains a JSON…
asked 2023-09-22T11:01:30.3233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 19%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
Mehrsa
0
Reputation points
commented 2023-09-22T15:38:11.86+00:00
Amira Bedhiafi
4,071
Reputation points
1 answer
ADF - Error at sink side - Rest API- Key Content-Type conflicts in AuthHeaders/AdditionalHeaders
Hello, I am getting error as below when I am trying to use the copy activity which as source as Json file from container and sink as Rest API. I have defined auth token in linked service . Content Type header is defined in additional headers in…
asked 2022-09-26T11:59:52.21+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 33%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAY%3C/text%3E%3C/svg%3E)
Anurag Yadav
1
Reputation point
answered 2023-09-22T14:54:25.3266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 75%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
Nadimpalli, R V Satya Avinash Varma
0
Reputation points
1 answer
One of the answers was accepted by the question author.
How to clone/copy Azure Data Factory to another tenant?
My organization is moving to another Azure tenant. We have an Azure Data Factory with 50 pipelines, 40 datasets, 15 linked services and 5 triggers configured in the current tenant that needs to be available in the new tenant. I know that moving Azure…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 5%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
1 answer
While copying multiple files in for each file, i am getting AzureStorageOperationFailedConcurrentWrite issue. Even though folder and files names are diffrent
Using look-up activity i am trying to read a file which contains the api data information(with folder structure) . Next I used for each activity to loop those records. In for each activity i have placed a copy activity to copy those files as per folder…
asked 2023-09-22T10:30:23.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 56.00000000000001%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMP%3C/text%3E%3C/svg%3E)
Mohan Parashetti Veerabhadrappa
0
Reputation points
edited an answer 2023-09-22T12:50:34.4633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 12%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
Suba Balaji
8,386
Reputation points
1 answer
Unable to replace valid phone number
https://examregistration.microsoft.com/ I am unable to register the exam as it fails to update my valid number exists in USA "The Phone Number you provided is invalid, please provide a valid phone number and try again."
asked 2023-09-18T15:49:46.7433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 77%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDN%3C/text%3E%3C/svg%3E)
Divya Nagarajan
0
Reputation points
commented 2023-09-22T12:35:05.7566667+00:00
Suba Balaji
8,386
Reputation points
1 answer
How to create single dataset for multiple azure data flows?
I have about 50 files that i need to transform using a Azure Data Flows. Is there a way i can create a single data set for multiple data flows? I want to avoid creating a dataset for every single file.
asked 2023-09-19T13:47:03.2733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 57.00000000000001%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPM%3C/text%3E%3C/svg%3E)
Pankaj Mandania
21
Reputation points
commented 2023-09-22T11:43:06.1133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
Harishga
90
Reputation points Microsoft Vendor
0 answers
Error when using Window activity in Azure Data Factory Data Flow
Hi, I am currently attempting to use the window activity in a data flow to recreate partition over order by. What I want is for the activity to rank rows with the same RECID and DML_Action by their LastProcessedChange_DateTime value, with the most recent…
asked 2023-09-22T11:38:00.53+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 84%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETR%3C/text%3E%3C/svg%3E)
Tobias Rognstad
0
Reputation points
asked 2023-09-22T11:38:00.53+00:00
Tobias Rognstad
0
Reputation points
1 answer
ADF -- Extract from SAP with RFC table option, filter on 2 or more columns
Hello Is there any way we can use multiple conditions on the RFC Table Option field of a copy activity? Something like this: Doing this on 1 column works. However having multiple conditions doesn't seem to work. I've seen that 'AND' operator or 'OR'…
asked 2023-09-13T18:17:24.0766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 34%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYS%3C/text%3E%3C/svg%3E)
Yannick Seynhaeve
0
Reputation points
commented 2023-09-22T11:16:31.63+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
165
Reputation points Microsoft Vendor
0 answers
can we bulk copy and migrate mysql database tables using azure data factory
i know we can bulk copy SQL database tables using for each in azure data factory, but can we do the same for mysql database tables?, if it is possible please suggest some solutions.
asked 2023-09-22T06:06:29.24+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 74%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPP%3C/text%3E%3C/svg%3E)
piyush pant
0
Reputation points
commented 2023-09-22T10:32:11.44+00:00
piyush pant
0
Reputation points
1 answer
Azure Data Factory - Copy activity using REST connector - Unable to access response headers
Hi team, I use ADF pipeline comprising copy activity to pull data from a remote endpoint using REST connector and then place the data on the blob. I do not have a need to use the pagination feature available with REST connector as the response is not…
asked 2023-07-31T03:06:55.41+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 50%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBK%3C/text%3E%3C/svg%3E)
Balachandran, Karthikeyan
0
Reputation points
commented 2023-09-22T09:40:19.0366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 87%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECS%3C/text%3E%3C/svg%3E)
Chandra Sekhar Katabathina (External)
0
Reputation points