Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,836 questions
0 answers
CI/CD Pipeline ADF Environments
Hi, I want to use CI/CD pipeline to deploy by ADF dev instance code to other environments(UAT, Prod). Can I deploy the code without integration runtime,linked services? I want to deploy only the rest of the code. As these IRs and integration runtimes…
asked 2024-06-07T05:23:48.5033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(19.2, 54%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESD%3C/text%3E%3C/svg%3E)
Santhi Dhanuskodi
185
Reputation points
asked 2024-06-07T05:23:48.5033333+00:00
Santhi Dhanuskodi
185
Reputation points
1 answer
Retrieving user sign-in status using Video Indexer API
Is there a API for the Video Indexer to check the sign-in status of a user for the Video Indexer ARM account? I'm looking for an API that returns this information for developers. Thank you for your help!
asked 2024-06-06T12:28:05.9866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 74%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVA%3C/text%3E%3C/svg%3E)
Vishnoi, Astha
160
Reputation points
commented 2024-06-07T04:59:04.2866667+00:00
Vishnoi, Astha
160
Reputation points
1 answer
One of the answers was accepted by the question author.
I have enabled ADF Studio preview feature - but I cannot see the Preview Semantic Model Refresh Activity
I see this is a new feature that I want to test out -- but I can't see it from ADF even though I have the preview update and preview experience turned on for my ADF. How do I see this and interact with this new preview activity? Thank…
asked 2024-06-05T21:47:30.1533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 35%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELH%3C/text%3E%3C/svg%3E)
Laura Hinchey
0
Reputation points
commented 2024-06-07T04:08:57.94+00:00
PRADEEPCHEEKATLA-MSFT
81,151
Reputation points Microsoft Employee
0 answers
CI/CD Pipeline ADF Environments
Hi, I have a single repository for ADF. All my code is developed in DEV ADF instance, And I have multiple uat and prod environments. I want to deploy the code in UAT and Prod using CI/CD Pipeline. During CI/CD pipeline deployment, Can we customize the…
asked 2024-06-07T04:08:08.2833333+00:00
Santhi Dhanuskodi
185
Reputation points
asked 2024-06-07T04:08:08.2833333+00:00
Santhi Dhanuskodi
185
Reputation points
1 answer
File movement from sharepoint Nested Subfolders using Microsoft Graph API to blob using ADF
I am trying to copy files from sharepoint to Azure Blob Storage using Microsoft Graph API in ADF. I am able to do it when the sharepoint folder structure is flat. I am facing issue when the sharepoint has nested subfolders. Is there a way to get…
asked 2024-06-05T02:34:02.96+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 95%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJS%3C/text%3E%3C/svg%3E)
Jagdev Sinder
0
Reputation points
commented 2024-06-07T03:22:46.4866667+00:00
Jagdev Sinder
0
Reputation points
0 answers
ADF web activity payload is to big
We are using ADF now, there is a step need send Post to Remote Sever, but this payload is too big, far more than 896KB, it's already 15MB, so we try to store the request payload to blob and then use the Copy Activity, but we facing another issue is Copy…
asked 2024-06-07T03:06:18.0733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 71%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERC%3C/text%3E%3C/svg%3E)
Ricky chen （陈帅）
0
Reputation points
asked 2024-06-07T03:06:18.0733333+00:00
Ricky chen （陈帅）
0
Reputation points
2 answers
getting error in copy data activity while using upsert
the pipeline is { "source": { "type": "DelimitedTextSource", "storeSettings": { "type": "AzureBlobStorageReadSettings", "recursive": true, …
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 32%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
0 answers
Copying files from sharepoint to blob
Hello @Anonymous I have three CSV files in the SharePoint folder (Shared Documents/Test/Invoices) that I want to copy to a blob storage, orders/invoices/PreProcessedinvoices In the web activity for "Get SPO Folder Metadata," I have the URL…
asked 2024-06-06T18:59:12.2366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 28.000000000000004%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKO%3C/text%3E%3C/svg%3E)
Kayode Ogidan
60
Reputation points
commented 2024-06-07T00:53:12.0133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
Harishga
4,640
Reputation points Microsoft Vendor
0 answers
ADF dynamic expression builder to keep trailing zeros on
I working with an API which accepts datetime in a particular format like yyyy-MM-ddThh:mm:ss.fffZ for example 2021-02-30T15:30:30.070Z when I write this back to cosmosDB to store this date for incremental load for next run, the training zero is not…
asked 2024-06-06T23:36:44.9533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(96, 33%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPU%3C/text%3E%3C/svg%3E)
Pavithra Udayakumar
0
Reputation points
edited the question 2024-06-06T23:38:16.8633333+00:00
Pavithra Udayakumar
0
Reputation points
0 answers
Synchronizing a Synapse table from an on-premise Oracle table
Hi, I've a Synapse table on a dedicated SQL pool and I need to update it reading from an Oracle table. I've already implemented a Self-hosted IR and I've already read from the Oracle table rightly with the Oracle connector. The requirement from my…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 41%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
2 answers
Having error in all my ADF pipeline with: InternalServerError executing request
Hi, Our pipeline in ADF are now having issue " Hit unexpected exception, please retry. If the problem persists, please contact Azure Support". I search this issue related to internal of Microsoft. Pls help to fix it asap. Thank you!
asked 2024-06-05T11:29:35.0866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 34%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPT%3C/text%3E%3C/svg%3E)
Phan Thi Minh Tho
0
Reputation points
commented 2024-06-06T17:59:35.2766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
11,125
Reputation points Microsoft Vendor
1 answer
Issue with Copy Dataverse data into Azure SQL using Synapse Link Template
I am experiencing issues using ADF I am trying to use the template "Copy Dataverse data into Azure SQL using Synapse Link" to move f&o data from datalake to Azure…
asked 2024-05-30T07:08:10.6766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 20%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDO%3C/text%3E%3C/svg%3E)
Dipo Onibile
0
Reputation points
edited a comment 2024-06-06T17:03:07.0733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
7,190
Reputation points Microsoft Vendor
0 answers
access error in purview
Hey, have registered Azure database connection, in purview but its gives an error can no access account I am the owner of the purview and sql database... given rights in IAM in db and purview but still same error
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
0 answers
Can i copy data from ADL to Neo4j graph database using ADF
Hello, I want to load data from ADL to neo4j database using ADF. Can I do that using ADF?? If yes, what are the different options I have? My first priority is I want to use ADF activities if that's not possible i can use python coding.
asked 2024-06-06T14:27:54.4+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 73%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Siddartha Reddy Jammula
20
Reputation points
asked 2024-06-06T14:27:54.4+00:00
Siddartha Reddy Jammula
20
Reputation points
0 answers
How to retrieve the number of signed-in users in ARM Video Indexer account?
Is there an API or any way to obtain the number and details of users that are currently signed in to an ARM Video Indexer account? I am getting this error message when trying to view a video from the link of video indexer account. In this video…
asked 2024-06-06T13:46:33.44+00:00
Vishnoi, Astha
160
Reputation points
asked 2024-06-06T13:46:33.44+00:00
Vishnoi, Astha
160
Reputation points
2 answers
One of the answers was accepted by the question author.
Export JSON string but exclude key-value pairs with null/empty values
I'm attempting to output some data originally in csv format from ADF as a JSON string but I need to exclude any key-value pairs where the value is null or empty (e.g. "Report": []). First I am using an Aggregate transformation to convert 3…
asked 2024-05-30T20:35:28.12+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 54%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDL%3C/text%3E%3C/svg%3E)
David Lang
40
Reputation points
accepted 2024-06-06T13:21:14.8833333+00:00
David Lang
40
Reputation points
1 answer
Issue with Merging Avro Files in ADF Pipeline
I have an ADF pipeline that copies data from blob storage to Data Lake Gen2. The pipeline merges multiple Avro files into one file for a given date period. I'm encountering an issue with the 'media_urls' column because its format is System.Array, which…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(240, 32%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EG%3C/text%3E%3C/svg%3E)
0 answers
Calling Power BI reports from ADF
Hello, I have one report with multiple client folders this is the ADLS gen 2 folder location for one client and data source connection: AzureStorageDataLake for different files: …
asked 2024-06-06T11:57:54.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 1%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBA%3C/text%3E%3C/svg%3E)
Bodke, Anil
0
Reputation points
asked 2024-06-06T11:57:54.79+00:00
Bodke, Anil
0
Reputation points
1 answer
Error in connecting Azure SQL Managed instance
Hi All, I am using copy activity to copy a table from On-Prem SQL Server (Source) to Azure SQL Managed Instance (Sink). I am getting the error as…
asked 2024-06-04T07:25:57.9366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 34%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
Mohanraj
150
Reputation points
commented 2024-06-06T11:05:04.3366667+00:00
Smaran Thoomu
11,125
Reputation points Microsoft Vendor
2 answers
I am encountering an error when trying to debug a data flow in Azure Data Factory that connects to an Azure SQL Database. My linked service and dataset connections to the Azure SQL Database are working fine, and I can preview data without any issues.
I am encountering an error when trying to debug a data flow in Azure Data Factory that connects to an Azure SQL Database. My linked service and dataset connections to the Azure SQL Database are working fine, and I can preview data without any issues.…
asked 2024-05-21T02:20:06.9066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 82%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERB%3C/text%3E%3C/svg%3E)
ram boyapati
15
Reputation points
commented 2024-06-06T08:57:30.9633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 4%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAC%3C/text%3E%3C/svg%3E)
Anoop Chundarathil
0
Reputation points