10,740 questions with Azure Data Factory tags

Sort by: Updated
0 answers

branch movement

How can we move branch from 1 repo to another repo... from an example Branch1 move to devrepo-- UATRepo Branch1 Please share screenshot...to move it manually without codings

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-10-19T15:24:32.0833333+00:00
Vineet S 950 Reputation points
0 answers

Azure Data Factory Question

Azure Data Factory Question Hi, I want to extract data from an API, and I'm confused about which activity in ADF I should use. Should I go with COPY or WEB activity? Can anyone please help?

Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
1,031 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-10-19T13:54:52.2833333+00:00
Ayaan Shaikh 0 Reputation points
1 answer

Azure Container App internet outages

Hi everyone! I'm struggling with a problem and could use some advice. I have a Python application that acts as a typical data sink, listening to data from a websocket provided by an exchange. The program runs in Docker, with the image stored in Azure…

Azure Container Registry
Azure Container Registry
An Azure service that provides a registry of Docker and Open Container Initiative images.
461 questions
Internet Information Services
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,876 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
Azure Container Apps
Azure Container Apps
An Azure service that provides a general-purpose, serverless container platform.
431 questions
asked 2024-10-18T23:12:34.4366667+00:00
Daniel Lasota 0 Reputation points
answered 2024-10-19T11:17:15.28+00:00
Vinodh247 21,881 Reputation points
1 answer

Azure Data Factory SFTP Linked Service: Failed to read binary packet data! (ProtocolError)

Hi everyone, I'm facing a problem with a SFTP server, where some restrictions are applied to avoid the usage of RSA Keys with SHA1 signature. I can connect without any problem to the SFTP server using WinSCP. There are no network restrictions so I can…

.NET
.NET
Microsoft Technologies based on the .NET software framework.
3,877 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-06-17T09:23:48.5933333+00:00
Paul Hernandez 666 Reputation points Microsoft Employee
commented 2024-10-19T10:41:52.88+00:00
Ashwin Antony 0 Reputation points
2 answers One of the answers was accepted by the question author.

How to Set Proper Time in Azure Data Factory Trigger (numbers are not viewable properly)?

Hi Team, When I trying to schedule a trigger. It seems there is issue.. The time is not properly visible. If you observe in below visual, its says 5. 5. 5... this is not clear. Is the bug or existing issue? Or i am missing something?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-10-18T03:31:45.8466667+00:00
Santosh J 20 Reputation points
edited an answer 2024-10-19T04:39:54.8066667+00:00
Santosh J 20 Reputation points
1 answer

Microsoft Fabric Data Factory - MS Graph API - Groups/Users - Unable to reference response paging attribute

I am trying to get all groups and users from MS Graph. These both use paging (limit 100 records per page). To get the next page I need to reference an attribute in the previous repsonse. However, the attribute has '@' and '.' in it... how do I reference…

Microsoft Graph
Microsoft Graph
A Microsoft programmability model that exposes REST APIs and client libraries to access data on Microsoft 365 services.
12,113 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
Microsoft Fabric Training
Microsoft Fabric Training
Microsoft Fabric: A Microsoft unified data platform.Training: Instruction to develop new skills.
46 questions
asked 2024-10-18T14:59:57.8433333+00:00
Laron Lemon 0 Reputation points
answered 2024-10-18T20:43:35.7166667+00:00
Amira Bedhiafi 25,261 Reputation points
1 answer

Azure Data Factory : How to set the bucket region of an Amazon S3 Compatible data source

Hi there, I'm trying to connect my Supabase storage, an Amazon S3 compatible storage to my Azure Data Factory for a copy flow. I'm able to connect to it with tools like Cyberduck, but it does not work with ADF. I guess the 403 error is related to the…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-10-18T10:00:18.4833333+00:00
Aurélien L 0 Reputation points
answered 2024-10-18T20:38:28.0633333+00:00
Amira Bedhiafi 25,261 Reputation points
1 answer

ADF Data Flow Mapping shows 0 rows written to Dataverse sink

Running a simple set of pipelines with data flow mappings to Dataverse sinks. Data flow runs successfully, but Rows Written through interface consistently shows 0 rows. I can see the rows were written by going to the table in Dataverse.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
Microsoft Dataverse Training
Microsoft Dataverse Training
Microsoft Dataverse: A Microsoft service that enables secure storage and management of data used by business apps. Previously known as Common Data Service.Training: Instruction to develop new skills.
34 questions
asked 2024-09-26T16:19:01.5633333+00:00
Noah Gilbert 26 Reputation points
commented 2024-10-18T20:27:58.07+00:00
Noah Gilbert 26 Reputation points
1 answer

Facing error while posting data to api using web activity

Post is working fine in postman but getting below error in synapse web activity Error calling the endpoint. Response status code: 'NA - Unknown'. More details: Exception message: 'NA - Unknown [ClientSideException] An error occurred while sending the…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,935 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2023-04-12T15:24:31.5333333+00:00
Lakshmi Niharika Paladugu 15 Reputation points
commented 2024-10-18T19:30:43.2633333+00:00
Patel, Siddhartha 0 Reputation points
1 answer

Convert to tab file

Hi am running activity and converting table from SQL server to .tab file but it contains 20 lacs records how can I convert this in a minute. Pls share screenshot

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-10-18T09:36:27.99+00:00
Vineet S 950 Reputation points
edited an answer 2024-10-18T19:22:06.3333333+00:00
Amira Bedhiafi 25,261 Reputation points
1 answer

No code found for {source} ~> {derivedColumn}

Hi, I created a flowlet in ADF which reads various sources and does appropriate transformations. This flowlet is called from a data flow and when previewing data, I get the error "No code found for stream DataFile ~> derivedColumn1". The…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-10-18T13:07:57.0133333+00:00
Bansal, Nimish 0 Reputation points
answered 2024-10-18T19:15:06.69+00:00
Amira Bedhiafi 25,261 Reputation points
1 answer

Shutdown hooks cannot be modified during shutdown

What does the ADF pipeline error message mean? 'Shutdown hooks cannot be modified during shutdown'.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-10-18T16:34:23.73+00:00
Alan Waddington 0 Reputation points
answered 2024-10-18T18:49:28.5433333+00:00
Amira Bedhiafi 25,261 Reputation points
1 answer

Main Repo in ADF

How to set up main repo between Dev and Accptance envionment in ADF. Genrally it should be same repo or diffrerent ...please share screenshot any

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-10-16T18:03:42.5633333+00:00
Vineet S 950 Reputation points
commented 2024-10-18T18:09:18.3+00:00
Chandra Boorla 2,200 Reputation points Microsoft Vendor
0 answers

Azure Data Factory: Copy from Blob to Blob Container with Hierarchical Namespace Fails with 409 Conflict

I am trying to copy data from one storage account to another. I have a pipeline with an activity that copies data from source storage account to sink. The pipeline runs successfully and I can see data has been copied. But, as soon as I try to copy the…

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
3,183 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-10-17T21:22:59.98+00:00
Noman Yaqub 0 Reputation points
commented 2024-10-18T17:56:06.1566667+00:00
Chandra Boorla 2,200 Reputation points Microsoft Vendor
1 answer

gnupg decrypting some files from an external sftp

Is it posible to decrypt some file than lands in an external sftp using ADF? Thanks

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-10-17T15:51:13.0666667+00:00
Jon Z 100 Reputation points
commented 2024-10-18T17:54:10.2666667+00:00
Chandra Boorla 2,200 Reputation points Microsoft Vendor
0 answers

ADF Snowflake New Connector Issues with KeyPair Authentication and Private Link

We are using Azure Data Factory (ADF) to connect to Snowflake with a Self-hosted Integration Runtime configured for Private Link. Recently, we migrated the Snowflake connector to the latest version that supports Key Pair authentication. Currently, Basic…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-10-18T09:55:11.7766667+00:00
Datta, Vinay 0 Reputation points
commented 2024-10-18T15:54:18.6066667+00:00
Smaran Thoomu 16,230 Reputation points Microsoft Vendor
2 answers

No code found for stream [nameofstream1] ~> [nameofstream2]

Hello I'm doing a preview on a data flow that is calling a flowlet. There is no validation error on the flowlet and all the modules display correctly but when I try preview I get the above error. Does anyone know what it could be?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-10-09T07:00:38.5666667+00:00
V 0 Reputation points
commented 2024-10-18T15:23:21.9466667+00:00
Smaran Thoomu 16,230 Reputation points Microsoft Vendor
6 answers

SAP CDC and SAP Tables connectors regarding SAP Note 3255746 ?

Dear Team, Is there any one who when through the SAP Note 3255746 ? What impact shroud we expect regarding the SAP CDC and SAP Tables connectors already implemented for our customers? Thanks for you input. Tarik

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,935 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-04-23T11:45:42.5766667+00:00
Tarik Zaaboul 5 Reputation points
commented 2024-10-18T15:11:01.93+00:00
Christopher Grancher 0 Reputation points
0 answers

Configuration for Transferring JSON from Azure Cosmos DB to Snowflake Using Azure Data Factory

We need to transfer an entire JSON object from Azure Cosmos DB to a Snowflake table. We have created a table in Snowflake with a single column named emp_details of type VARIANT to store the JSON data. How can we configure the mapping in Azure Data…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
asked 2024-10-15T09:40:05.8+00:00
Sowndar Kumaresan 0 Reputation points
commented 2024-10-18T10:54:05.7666667+00:00
phemanth 10,740 Reputation points Microsoft Vendor
1 answer One of the answers was accepted by the question author.

Time Zone Parameter Ignored in Azure Data Factory PostgreSQL Connector After Migration

I'm experiencing an issue with Azure Data Factory (ADF) when using the PostgreSQL connector. Problem: The "Time Zone" parameter in the Linked Service appears to be ignored, and timestamps are always retrieved in UTC, despite attempts to set…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,740 questions
Azure Database for PostgreSQL
asked 2024-10-11T11:52:38.2166667+00:00
icatii 25 Reputation points
accepted 2024-10-18T07:42:48.2266667+00:00
icatii 25 Reputation points