10,137 questions with Azure Data Factory tags

Sort by: Updated
1 answer One of the answers was accepted by the question author.

Azure Data Factory error creating blob

Hi, I have a pipeline in Azure Data Factory that queries Cosmos DB for some data and writes this data in a blob storage. It was working until it wasn't. I changed the query in the copy activity for a specific scenario, and when I ran it again it failed…

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,916 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,618 questions
Azure Cosmos DB
Azure Cosmos DB
An Azure NoSQL database service for app development.
1,535 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-20T12:55:13.6333333+00:00
Joao Borba 20 Reputation points
commented 2024-07-22T04:56:20.58+00:00
Vinodh247 13,301 Reputation points
1 answer

How to generate token for client_assertion without certificates?

We have an app registration which is connected to an external API. We have to use this API in our Azure Data factory. For this purpose, till now we were generating client secrets and generating tokens (in Azure Data Factory) from this URL -…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
Azure App Service
Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
7,363 questions
asked 2024-07-19T11:14:11.81+00:00
DevanshAgarwal-0149 0 Reputation points
commented 2024-07-22T04:40:15.02+00:00
DevanshAgarwal-0149 0 Reputation points
1 answer One of the answers was accepted by the question author.

फोनपे पर पर शिकायत दर्ज कैसे करें?

इस ऑफर के लिए ग्राहक से संबंधित सभी प्रश्न फोनपे/care या 09831-228932 पर पूछे जा सकते हैं...

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-22T03:25:36.7033333+00:00
accepted 2024-07-22T03:25:47.9766667+00:00
2 answers One of the answers was accepted by the question author.

मैं फ्लिपकार्ट से शिकायत कैसे कर सकता हूं?

अपनी शिकायत दर्ज कराने के लिए फ़्लिपकार्ट के टोल-फ़्री ग्राहक सेवा नंबर पर कॉल करें। फ़्लिपकार्ट ग्राहक सेवा अधिकारी से बात करने के लिए 08167-8298-78 या (081)67829-878 डायल करें। आपको अपनी समस्या को स्पष्ट और संक्षिप्त तरीके से बताना होगा।

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-22T03:24:35.69+00:00
Fggggg Llkkjj 100 Reputation points
accepted 2024-07-22T03:24:56.8566667+00:00
Fggggg Llkkjj 100 Reputation points
0 answers

Data Factory monitoring by inserting data table

Hello, I would like to know the best way to insert Datafactory activity logs into my Databricks delta table, so that I can use dashbosrd and create monitoring in Databricks itself , can you help me? I would like every 5 minutes for all activity logs in…

Azure Monitor
Azure Monitor
An Azure service that is used to collect, analyze, and act on telemetry data from Azure and on-premises environments.
3,017 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,916 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,072 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-21T22:18:21.9966667+00:00
Yohanna de Oliveira Cavalcanti 200 Reputation points
0 answers

adf from configtable

Hi, how to load tablename from config table and using copy activity at front end... how it will execute it.... let say copy activtiy lookup for tablename then how copy activity will take that...please help with screenshot ...

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-21T20:01:20.1833333+00:00
Vineet S 425 Reputation points
1 answer

For each loop in adf

I am running for each loop using look up activity so using look up activity I am getting all the files from blob storage but for each loop is running continuously.. It has 60 files but after getting from files... It is running again... Please screenshot…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-18T16:11:55.0666667+00:00
Vineet S 425 Reputation points
edited a comment 2024-07-21T19:45:56.01+00:00
Vineet S 425 Reputation points
1 answer

ADF managed Airflow tasks fail without running or logs

Hi, I have a managed Airflow instance inside Azure Data Factory. I am seeing a behaviour on a semi weekly basis where tasks scheduled for running suddenly fail with no logs. Whenever I retry any task (regardless of how heavy or light it is). It instantly…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-21T12:41:08.4033333+00:00
Peter Yacoub 0 Reputation points
answered 2024-07-21T16:24:44.76+00:00
Vinodh247 13,301 Reputation points
1 answer

Internal Server Error while calling azure function from Azure data factory

We have written script to download some files from sharepoint to the azure blob storage. Ii have tested the code locally and on the portal too. This function will copy single files so we have placed azure function activity in a for each activity to run…

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
4,644 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-20T13:46:57.1066667+00:00
Mansi Yadav 40 Reputation points
answered 2024-07-21T16:11:28.1266667+00:00
Vinodh247 13,301 Reputation points
0 answers

DP-203 Labs

Is there any way to do all the labs for DP-203 without costing alot of money AND without taking like 10 minutes to install something? For each lab, the ./setup.ps1 file takes like 10 or 15 minutes to load. I'd like to not have to delete the resources…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,425 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,672 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,072 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
Azure Training
Azure Training
Azure: A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.Training: Instruction to develop new skills.
1,295 questions
asked 2024-07-21T00:03:12.49+00:00
Ben Cohn 5 Reputation points
commented 2024-07-21T16:00:38.5466667+00:00
Vinodh247 13,301 Reputation points
1 answer

Google drive on Azure without GCP

Hello, I'm wondering what options I have to transfer Google Drive data to Azure without GCP access. I found that the Google Drive API and Google Storage API require a Google Cloud account. What other options do I have to securely connect to and transfer…

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
4,644 questions
Azure AI Search
Azure AI Search
An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.
861 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-21T12:18:22.7166667+00:00
Tony Minozi 0 Reputation points
edited an answer 2024-07-21T14:28:49.01+00:00
Dillon Silzer 56,361 Reputation points
1 answer

Test Connection is getting failed in data factory for Mongo DB

Test Connection is failing in datafactory for Mongo DB. Note, Ips are whitelisted but i am getting the below error. Failed to operate data via MongoDB client. An exception occurred while opening a connection to the server. Exception has been thrown by…

Azure SQL Database
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-03-01T09:45:38.9033333+00:00
ADMMMLIFEADFADMIN 0 Reputation points
commented 2024-07-21T10:16:44.8366667+00:00
Vinodh247 13,301 Reputation points
1 answer

[Legacy Connector] ADF Copy-table from Salesforce (legacy) activity failed

Dear team, We have a pipeline has been running for few years, it copies from sales force org to an SQL table a list of objects. It has been running well, till recently in the middle of the lists , it hangs on different table without being able to…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-04-30T09:51:26.6733333+00:00
Vincenzo Dell'Oste 0 Reputation points
commented 2024-07-21T10:10:10.68+00:00
Vinodh247 13,301 Reputation points
2 answers

Error when loading SQL Server Views into JSON files during COPY DATA activity

I am copying data from SQL Server to Azure Blob Storage as a JSON file with gzip compression using a COPY DATA activity within a ForEach activity. However, if the table from the lookup activity is actually a view in SQL Server, I get an error "Index…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-04-08T02:04:41.8366667+00:00
Jun Hao Goh 0 Reputation points
commented 2024-07-21T10:09:04.97+00:00
Vinodh247 13,301 Reputation points
2 answers

Azure Data Factory Error- when trying to connect to Azure SQL Database. How to resolve? Why happened?

Spark job failed: { "text/plain":…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-05-04T18:37:19.4233333+00:00
Fatemeh Ghorbani Sini 0 Reputation points
commented 2024-07-21T10:08:20.5733333+00:00
Vinodh247 13,301 Reputation points
0 answers

My python code uses AzureCliCredential() function, but it is giving error in running in Synapse/ADF/Databricks notebook .

My python code uses AzureCliCredential() function, but it is giving error in running in Synapse Workspace that your Azure Cli command not found on path . I have tried using other function like DefaultAzureCredentials(), ClientSecretCredentials () also…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,672 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,072 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-18T18:55:12.8266667+00:00
Utsav Mori 20 Reputation points
edited a comment 2024-07-20T19:36:18.43+00:00
Utsav Mori 20 Reputation points
1 answer

The on-prem NAS drive copies data from multiple locations to the Blob Storage

I have a requirement to copy data from a NAS drive to blob storage using Azure Data Factory (ADF). The NAS drive has 51 different locations, examples of which are: T:\scd\in T:\vnd\ed\in T:\tx\rx\in\em T:\rx\gs\tm\in\em The NAS server is denoted by…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-19T15:03:28.6166667+00:00
Brahmendra Shukla 0 Reputation points
commented 2024-07-20T18:52:54.2133333+00:00
Brahmendra Shukla 0 Reputation points
2 answers

ADF Issue with partially uploading data to Cosmos DB SQL API

I am migrating data from one container to another container in Cosmos DB SQL API, where I only changed the partition key. However, when migrating using ADF, it is not writing the complete data at the sink side. Can you please guide me on how to resolve…

Azure Cosmos DB
Azure Cosmos DB
An Azure NoSQL database service for app development.
1,535 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-19T15:09:45.43+00:00
Brahmendra Shukla 0 Reputation points
commented 2024-07-20T18:50:20.27+00:00
Brahmendra Shukla 0 Reputation points
1 answer

ADF Copy Activity - Emply col issue

Hi Team, I have a pipeline where I am copying the data from the file to a database. The file had one extra empty column in between, and my copy activity failed with the below error: Message: Invalid Excel header with empty value It failed because the…

Azure SQL Database
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-20T02:14:27.9266667+00:00
ADF_Coder 0 Reputation points
commented 2024-07-20T14:55:08.76+00:00
ADF_Coder 0 Reputation points
3 answers

SQL Server wont connect to data factory

I recently set up a SQL server and database. However, when I try to connect it to data factory using SQL Authentication, i get the following error Cannot connect to SQL Database. Please contact SQL server team for further support. Server:…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,137 questions
asked 2024-07-19T12:52:59.1733333+00:00
Sage Desiderio 0 Reputation points
commented 2024-07-20T11:00:08.4133333+00:00
Vinodh247 13,301 Reputation points