Azure Data Factory

• -805 Error when pulling data from a DB2 data source in ADF
• 'Failed to encrypt sub-resource payload' error when attempting CI/CD
• "An existing connection was forcibly closed by the remote host" when using Azure Key Vault with Self-Hosted Integration Runtime
• "Error: The databricks job of Dataflow completed, but the runtime state is either null or still InProgress." - Error, even when the dataflow succeeded as expected.
• [ADF] : Write partitioned parquet files
• A database operation failed with the following error: Invalid object name inDataFromSource
• Access denied when using key vault secret for Linked Service
• Access SharePoint list data using ADF
• Accessing GitHub Repository in Data Factory is impossible due to invalid personal access token
• Accessing String Array from ADFv2 Split
• Activity dataflow1 failed: DF-SYS-01 at Sink
• Activity failed - property 'value' doesn't exist, available properties are 'firstRow, effectiveIntegrationRuntime'.
• Adding additional column in data being saved in CosmosDB by Azure data factory;s copy activity
• ADF - Data Flow seems to be super slow
• ADF - Encoding
• ADF (v2)- Read PDF and Image File
• ADF CI/CD exceeding maximum number of template parameters
• ADF connection to OneDrive.
• ADF Continuous Integration - DataLake fails if self hosted integration selected
• ADF Copy Activity JSON to SQL mapping error
• ADF Copy Activity: Do we have a way to select fewer columns out of total columns in BLOB source with CSV file mapping sql
• ADF Copy from File System to Azure Data Lake Fails with HybridDeliveryException
• ADF dataflow error
• ADF Failure: Function "Length" expects its parameter to be array or string
• ADF Integration Runtime issue with invalid Authentication Key
• ADF Rest Pagination w/ Relative URL
• ADF V2 - How to Check If Stored Procedure Output is EMPTY
• ADF V2 - Intermittent Error Eventually Resolves Itself "The remote name could not be resolved"
• ADF V2 - Is Email Notification (not using Logic App) Possible
• ADF v2 copy activity long queue time
• ADF V2 pull from REST API that returns XML
• ADF v2: Getting list of SFTP files with "Get Metadata" activity times out with large file count in folder
• ADF Warning on @string
• ADF-Check the Existence of Blob in Azure Storage Account
• ADF-cloud orchestration activity run cost ?
• ADF: The number of template resources limit exceeded
• ARM Template Custom Parameters for Integration Runtime
• ARM template deployment with shared self hosted integration runtime failing
• Azure Data Factory Kafka Connector
• Azure ADF - ContentMD5 field in Get Metadata activity is always null
• Azure Data Factory - Create Dynamic Sink/Destination Folder Structure
• Azure Data Factory - Get a List of Files but ignore folders
• Azure Data Factory - item() is not a recognized function
• Azure Data Factory - Polybase option to load Azure SQLDW error
• Azure Data Factory - REPLACE FUNCTION , pass previous activity result as TEXT and NEW TEXT
• Azure Data Factory -Publish Error
• Azure Data Factory Copy Activity on Failure | Expression not evaluated
• Azure Data Factory Copy Behavior Merge File issue
• Azure Data Factory does not fire SQL trigger on insertion
• Azure Data Factory Pagination with offset & limit paramters
• Azure Data Factory Pipeline failed while running data flows with error message : Hit unexpected exception and execution failed
• Azure data factory re-startability from the point of failure?
• Azure Data factory V2 - Remote Shared File Path Access Issue using Self Hosted IR
• Azure Data Factory V2 SFTP Throttling Error
• Azure Data Factory: How to use a footer/trailer record in a fixed-width text file to validate the row count of the file
• Azure data factory's Event trigger for pipeline not working for Blob Creation/ deletion most of the time
• Azure data flow not showing / in path to data source
• Azure DataFactory - How to prevent to execute multiple instances of same pipeline in parallel
• Azure: How to trigger ADF pipeline using powershell?
• AzureTable + DataFactory: azureTable Data Type Byte is not supported. Detailed
• Can I import multiple notebooks into databrick?
• Can't get lookup data passed to Oracle Database
• Can't use a query in Azure Data Factory Data Flow Cosmos DB Source
• Ceate Linked service for Sql Server with windows authentication in Azure Data Factory
• Column Not Found Error in Mapping Data Flow
• Connect Azure Data Factory to Online Microsoft Dynamics 365 Business Central
• Copy a pipeline from one Data Factory to another
• copy activities on Azure IR has long queue wait times
• copy activity :Error converting datatype from VARCHAR to DECIMAL
• copy activity additional headers issue
• Copy Activity Does Not All Wildcard File Name Expressions
• Copy Activity fails with "found more columns than expected column count"
• Copy Activity in Azure data factory do not copy multi line text in sql table maintaining the line breaks.
• Copy Activity is rounding off decimal data
• copy activity, REST dataset, additional headers
• Copy data activity - ErrorCode 2200 - UserErrorColumnNameMismatchByCaseSensitive
• Copy data activity and decimals (error 2200 Failure happened on Source side. Type = System.OverflowException, Message = Conversion overflows)
• Copying From Self-hosted IR SQL Server to Azure DataLake Gen 2 in Parquet files
• Could not find member 'metadata' on object of type 'DeploymentParameterDefinition'. Path 'properties.parameters.factoryName.metadata
• Could not load resource. Please ensure no mistakes in the JSON and that referenced resources exist.
• Create Dynamic Linked Service in Azure Data Factory V2
• Create Folder-Structure in Azure Data Lake Gen2 based on FileName
• Current Activity RunId
• Custom Activity Output values for use in sequential task
• Data connector for Azure Analysis Services in Data Factory
• Data factory - SFTP multifactor authentication
• Data factory pipeline is suddenly giving error
• Data Factory UI is not loading
• Data factory V2 - Error copying files with prefix from blob storage to data lak
• Data Factory V2 Copy Activity erroring Bulk Copy failed due to received an invalid column length from the bcp
• Data Factory V2 Error "errorCode": "2200", "message": "Failure happened on 'Sink' side.
• Data Factory V2 Http Request timeout
• Data Factory V2, Pipeline Activity "On Success" and "On Completion" Property Defference
• Data Factory: Connect Linked Service to Blob vis SASURI
• Data fetch Error in CosmosDB
• Data flow debug won't start
• Data Flow Upsert/sink not working - Key Column "columnName" does not exist on mapped columns
• Data flow: parquet source support types
• Data Flows Error: Detected implicit cartesian product for LEFT OUTER join
• Data issue copying data from ADLS delta lake folder to SQL Datamart
• Database operation failed. Error message from database execution : The given value of type String from the data source cannot be converted to type nvarchar of the specified target column..
• DataFactory v2 loading CRM - How to autoset GUID?
• Datetime milliseconds ignored by Azure?
• Default Sorting of Get MetaData Activity
• Delete Activity - clear files if they exist
• Delete Activity in Azure Data Factory
• Delimited text with thousand and decimal separator (fx 42.000,16)
• DF V2 - All Pipelines and Data Sets now saying - This resource references resources which could not be loaded
• During an Copy Activity, how to deal with null value?
• Dynamic Variable in URL in Azure Data Factory V2
• Dynamically pass table name to a SQL query in ADF
• Empty Body - Web Activity - A valid body is required for PUT and POST request
• Empty string should be treated as Null and Null value must be set to empty string
• Error creating linked service that references Azure Key Vault linked service
• Error interacting with REST API: Request had insufficient authentication scopes
• Error when copying from AWS S3 to Azure Blob using ADF
• Error When Trying Using New SFTP Sink Connector
• Error while copying data from ADLS Gen1 to SQLDW
• Error while copying data from azure sql to oracle
• Error while installing Self hosted IR on laptop
• Error; Table is required for copy activity
• ErrorCode 2108 when trying to make a POST Call to Azure SQL DWH
• Escape character in Azure data factory strings
• Event Trigger - Error when getting container list
• Exception UserErrorSourceBlobNotExist occurs on Hello world example of copy file with Azure Data Factory
• Execute shell script in Azure Linux VM using ADF
• Failed to save Arm Templates
• Failure - FailedDbOperation - Microsoft.DataTransfer.Common.Shared.HybridDeliveryException
• Filter activity inside for each activity
• Firewall issue while connecting to SQL server via Data Factory V2
• Get all pipelines and their run status using Azure PowerShell / API
• Get Child Items size
• Get Metadata Dynamic Content Output
• Get output of Get Metadata activity of Data Factory into a file
• Get the total running time of pipeline in Azure Data Factory
• Getting error Pipeline is not found, please publish first in ADF
• Getting Issues while copying data to Ownerid field in Crm using Copy Activity
• Handling fixed width files in Azure Data Factory
• How can i compress\uncompress (zip\unzip) multiple files using ADF.?
• How can I start the Databricks / Spark cluster from ADF?
• How do I iterate over each Row of a Table Storage using Powershell?
• How long does Azure Data Factory keep Pipeline run history
• How many parallel activities can be run in Azure data factory
• How to 'Get Latest' of my ADF pipeline from Azure Repos Repository
• How to calculate the difference between two dates in days (expression language)?
• How to connect AS400 with Azure DAta Factory !!
• How to connect with ADF to SQL Server on Azure VM?
• How to convert different date formats in data factory data flow?
• How to create condition on copy output property which may or may not exist
• How to do a Dynamic Column mapping in Copy Activity
• How to fix ADF creating empty files for folders
• How to Force Failure in ADF Pipeline using set variable activity or validation activity
• How to force stop a Azure Data Factory Pipeline Trigger
• How to get Count of records while using COPYDATA?
• how to get file Count in Azure data factory and do activities based on it
• How to get list of files from folder via Get Metadata activity in Data Factory
• How to get the blob storage file name that is getting processed inside ADF V2 pipeline?
• How to Increase Azure-hosted Integration Runtime JVM Heap Memory
• How to increase DIUs for Copy Data in Azure Data Factory and reduce the copy duration
• How to insert ConnectionString into a Key Vault
• How to iterate through files in Blob Storage in ADF V2
• How to obtain the Refresh Token for a Google Big Query Linked service
• How to Parameterise the file path for Azure Blob Storage
• How to pass a parameter to an Azure function in ADF v2
• How to pass parameter values from one Pipeline to another Pipeline?
• How to post data to a REST API in ADF?
• How to re-run a trigger by manual?
• how to read csv file with other encoding in azure Databricks
• How to Read files from many nested folders using Get Metadata Activity?
• how to reconfigure integration runtime (self-hosted) node ?
• How to rename a file in azure blob using azure data factory?
• How to rename a folder in azure data factory pileline
• How to replace values in data factory data flow?
• How to run single activity in azure data factory Pipeline
• How to skip the mapping of Identity columns in Copy Data Sink table?
• How to spilt the large json file in Blob?
• how to truncate table with different name than source table via pre-copy script
• How to use pipeline parameters as dataset parameters
• ImportError: No module named azure.storage.blob
• Increment a simple variable
• Increment Variable value
• Inner Pipeline Activity Outputs
• Inserting a file and folder path with dynamic content in DF V2
• Invalid Column Name error when trying to update a row using Data Flow sink
• Issue with SFTP connector
• Iterate Through a Source Folder to Copy/Delete Files Using Data Factory V2
• Loading Data Factory with csv file with commas in a column causing errors
• Look Up activity -Stored Procedure with Out param
• Lookup Activity failing with Error code 2100
• Lookup Error: The data type ByteArray is not supported
• Mapping Data Flow - Escaping single quote
• MD5 is missing when creating csv file from azure data factory copy pipe line?
• MergeFiles Copy Behaviour in CopyActivity in Azure Data Factory
• Multi-character column delimiter not supported in ADF connection.
• Multiple nodes in integration runtime
• Need 64 bit MySQL Connector/NET for ADF
• Need to Implement Loop Counter in foreach Activity
• Need to insert date in database column in while data ingestion in flat file to table on data factory
• Nested IF statements
• On-Premises File System error 'The network name cannot be found'
• Operation on target ForEach_1oy failed: Activity failed because an inner activity failed
• Operation returned an invalid status code 'Conflict
• Out of memory error in ADF copy
• out of memory in ADF
• Parquet file contained column 'errCol', which is of a non-primitive, unsupported type
• Pass output from one pipeline run and use as parameter in another pipeline
• Pipelines being triggered twice by Blob
• Polybase, empty string becomes NULL #sqldatawarehouse #polybase #datafactory #usetypedefault
• Post-copy script in the Copy Data task in Data Pipelines?
• Problem in Publishing in Azure Data Factory
• Process multiple Gen 2 folder to their respective SQL tables. Each folder can have one to many file which need to be processed.
• Publishing Error even with proper authorization
• Read nested array in JSON using Azure Data Factory
• Replace Data in ADF(Dataflow)
• Replace multiple values using expression language in azure data flow
• Rollback Failed Pipeline to avoid partial commit
• Secure string type Pipeline parameter
• Self Hosted Integration Runtime is unavailable while creating a LinkedSerive In AzureDataFactory pipeline
• Self hosted integration runtime setup - Minumum permission required to start the service
• Send Email using Azure function without using third party tool
• SFTP as sink - getting can't find SFTP path error
• SFTP connection failed while reading private key from keyvault but it works fine while reading the key content directly from the key file
• SFTP connection failing with error : Failed to connect to Sftp server 'xx.xx.xxx.xxx'. Server response does not contain SSH protocol identification.
• Skipping Rows on Data Factory Dataset or Pipeline
• Source and sink in the copy activity must be connected via the same self-hosted IR
• Specify Order for Items in Foreach Activity
• SqlDecimal from the data source cannot be converted to type decimal of the specified target column
• Stored procedure in ADF issue
• Substring() and Length()
• Table Storage Source with nullable datetime property does NOT work if property value of first table entry is null
• Table Type and Parameter in ADF with SQL Database as Sink
• The argument must not be a empty string.
• The client has permission to perform action 'Microsoft.EventGrid/EventSubscriptions/Write' on scope XXX however the linked subscription 'subscriptions' was not found.
• The datetime string must match ISO 8601 format - at azure data factory pipeline - getting error randomly
• The Format of Body for a POST request in Web Activity in Azure Data Factory
• The integration runtime (self-hosted) node has encountered an error during registration
• The SELECT permission was denied on the object ADF Pipeline
• The term 'Get-AzDataFactoryV2Trigger' is not recognized as the name of a cmdlet
• Trigger ADF data pipeline from SFTP/FTP location
• Unable to connect to Oracle database
• Unable to create a new linked service (File System)
• Unable to Delete linked Service - Reference By a Dataset
• Unable to evalute the expression in the parameter of the pipeline
• Unable to extract data from REST API by passing Multiple parameters in ADF
• Unable to have a validation activity inside an IF activity
• Unable to insert into Snowflake using Azure Data Factory V2 with Snowflake ODBC (V2.20) 64 bit - System DSN on Windows VM
• Unable to parse expression for dataset parameter
• Unable to use the Pagination Rules in REST sources in ADF
• Unble to connect to on-premise DB2 using Azure Data factory
• Unexpected character encountered while parsing value
• Until loop does not break when an inner activity errors
• UnZip File( s) and Copy Into Azure Data Lake in ADF
• Use of Ticks() function
• Use variable for querying in SQL source in Copy Data activity
• Using wildcards in datasets and get metadata activities
• Validation Activity issues
• Variable value write to blob text file?
• Webjob via WebHook in Azure Data Factory
• What are the file size limits for ADF.?
• What should I add in Azure Data Factory Webhook body and how can I generate or get Callbackuri?
• Working with Zip Files in Azure Data Factory