Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,638 questions
1 answer
Azure DevOps pipeline suddenly start failing!
2024-07-09T23:11:19.7718791Z Complete getting Artifacts From Template 2024-07-09T23:11:19.7719155Z Start deploying artifacts from the template. 2024-07-09T23:11:19.7722609Z Deploy dev_keyVault_linkedService of type…
asked 2024-07-09T23:45:33.4466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 19%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Sultan Khan
0
Reputation points
answered 2024-07-10T05:23:51.67+00:00
AnnuKumari-MSFT
31,911
Reputation points Microsoft Employee
0 answers
Azure Data Factory Copy Geometry data from Azure SQL to Azure Cosmos DB
Hi Team, I am trying to copy over data from a table in Azure SQL database containing geometry type columns. When i created a linked service and preview data get the error message 'Type=Newtonsoft.Json.JsonSerializationException,Message=Error getting…
asked 2024-07-10T05:11:28.7133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 70%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVR%3C/text%3E%3C/svg%3E)
Vijay Rudrasamudram
0
Reputation points
asked 2024-07-10T05:11:28.7133333+00:00
Vijay Rudrasamudram
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Static Code Analysis for ADF or Synapse Pipelines in Azure
Hi team, I would like to know is there any use case possible to perform static code analysis of our ADF or Synapse pipelines residing in Azure DevOps repository? Thanks, Debajit Mitra
asked 2024-06-14T03:42:29.93+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 48%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDM%3C/text%3E%3C/svg%3E)
Debajit Mitra
20
Reputation points
accepted 2024-07-10T04:04:11.9866667+00:00
Debajit Mitra
20
Reputation points
0 answers
check a field value is not null in derived transformation in dataflow
I am checking if a field(column) has data that is not NULL. How do i use NOT ISNULL(field1) in derived transformation ? I want to convert following code in derived transformation example: iif( not isNULL(field1), field1, iif( not isnull(field2), field2,…
asked 2024-07-09T17:16:30.26+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 23%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPS%3C/text%3E%3C/svg%3E)
Peruka, Sikander reddy
65
Reputation points
commented 2024-07-10T03:29:16.0333333+00:00
Peruka, Sikander reddy
65
Reputation points
0 answers
Accessing Fabric Lakehouse via Logic App
Has anyone had any luck connecting a logic app to a lakehouse in Microsoft Fabric? Some additional background: We have a data pipeline that is ultimately generating a summary of some actions taken within the pipeline. This summary is stored as an Excel…
2 answers
One of the answers was accepted by the question author.
AzureDF IfCondition in ForEach activity not acting as expected
Hi all, I am new to Azure and I am testing on data factory. I have a container which stored three CSV files. I would like to build a pipeline that copy the files to different output container base on the filename. I added a Get Metadata activity and…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 65%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EB%3C/text%3E%3C/svg%3E)
1 answer
Why is data not getting inserted in DB after the data flow task is successful in ADF
I have a dataflow task which is supposed to insert data in a DB. Im using SQL Server as DB and am trying to insert data in a table. When we execute the pipeline on ADF which has the required data flow task, we can see that the pipeline executes…
asked 2024-02-15T11:30:09.68+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 45%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EQM%3C/text%3E%3C/svg%3E)
Quadri, Mumin Ahmed
0
Reputation points
edited an answer 2024-07-09T16:58:37.5466667+00:00
Quadri, Mumin Ahmed
0
Reputation points
1 answer
Use yaml file to create Azure Dashboards
Can we use yaml file to create Azure Dashboards. Yaml file can contain some kind of data quality validations.
asked 2024-07-09T12:16:30.68+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 34%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHV%3C/text%3E%3C/svg%3E)
Harshita Vishwakarma
45
Reputation points
answered 2024-07-09T16:18:31.5266667+00:00
Clive Watson
5,951
Reputation points MVP
1 answer
Getting the size of parquet files from azure blob storage
I have a blob container abcd The folder structure is like below: abcd/Folder1/Folder a, Folder b…..Folder z Inside a particular Folder a/v1/full/20230505/part12344.parquet Similarly Folder b/v1/full/20230505/part9385795.parquet Scenario is I need to get…
asked 2024-07-09T11:57:50.94+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 89%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKJ%3C/text%3E%3C/svg%3E)
KEERTHANA JAYADEVAN
66
Reputation points
edited the question 2024-07-09T16:00:08.9+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 76%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV%3C/text%3E%3C/svg%3E)
VarunTha
5,355
Reputation points Microsoft Vendor
1 answer
Could someone from Microsoft provide details on SAP Note:-3255746
Hello Team, Could you please provide an insight on ODP context in Synapse Analytics and that ABAP_CDS cannot be use directly through this methodology after SAP Note? If we have to expose SAP Data it has to be through only Odata? is that correct? Have…
asked 2024-07-08T20:38:48.46+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 22%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYP%3C/text%3E%3C/svg%3E)
Yash Parikh
0
Reputation points
commented 2024-07-09T11:23:52.21+00:00
Yash Parikh
0
Reputation points
0 answers
Can we remove the artificial limitation of ADF Snowflake V2 supporting Multiple Statements
Issue with Snowflake V2 https://learn.microsoft.com/en-us/azure/data-factory/connector-snowflake?tabs=data-factory#differences-between-snowflake-and-snowflake-legacy How can I get feedback to the engineers that are working on this new connector for…
asked 2024-06-25T06:37:47.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 81%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMN%3C/text%3E%3C/svg%3E)
Macolino, Nick (Health)
1
Reputation point
commented 2024-07-09T10:43:53.1766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
12,185
Reputation points Microsoft Vendor
2 answers
Writing to the sink failing with error: Invalid column name 'true'
Hi, One of our dataflow is failing in production with following error: 'Job failed due to reason: at Sink 'sink1': Invalid column name 'true'. Invalid column name 'true'. The dataflow inserts records to a Synapse SQL server table. This was working…
asked 2024-06-26T13:31:00.6866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 33%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
Subedi, Manish
40
Reputation points
commented 2024-07-09T10:36:46.31+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 72%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFM%3C/text%3E%3C/svg%3E)
Florian Menzel
0
Reputation points
0 answers
primary key in adf pipeline
ADF package is failing due to Primarky and not null column available in target table due to which pipiline is failing how to handle such situation in adf.. Please prrimary key is only creating unique constraint
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
1 answer
multiple lookup in copy activity
Hi, how to handle multiple lookup column as input in copy activity
asked 2024-07-06T02:27:07.44+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Vineet S
305
Reputation points
commented 2024-07-09T08:59:00.3+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
75
Reputation points Microsoft Vendor
1 answer
Some Rows are getting dropped while sinking in CSV format in Azure Data Factory using Data flows
I am using csv as a row, and tranforming it in Azure Data flow in Pipeline. while sinking it in CSV format, some rows are getting dropped, but they are present in data preview when I am adding filter condition to look that particulat ID.
asked 2024-07-09T06:18:37.7666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 60%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Shivam Kumar
0
Reputation points
answered 2024-07-09T08:30:48.2533333+00:00
Amira Bedhiafi
18,741
Reputation points
2 answers
Migration to Azure
Hi friends, we have a requirement to Migrate Denodo to Azure, and I have these questions: What could be the challenges we can anticipate and be aware of? Does the data's size impact the migration's complexity and duration? What tools are the best suited …
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 15%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
1 answer
ADF to execute dynamic databricks notebook
Hello, in ADF, I have a pipeline which executes a databricks notebook. In the settings tab of the notebook in ADF, in the notebookPath I have placed the path as follows: /Users/myusername/company/xyz/notebookName The above path works fine and the ADF…
asked 2024-07-09T05:38:58.4266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 97%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
arkiboys
9,686
Reputation points
edited an answer 2024-07-09T07:00:21.9966667+00:00
Smaran Thoomu
12,185
Reputation points Microsoft Vendor
1 answer
Dynamic sql server table name inside copy activity
There is a copy activity inside the ADF pipeline which in the source tab has the linked service to sql server and the table option is selected and in the query section I have placed select * from tbl_company This works fine Now I want to make this table…
asked 2024-07-09T05:48:31.91+00:00
arkiboys
9,686
Reputation points
answered 2024-07-09T06:15:06.1+00:00
Chandra Boorla
75
Reputation points Microsoft Vendor
1 answer
How to flatten nested json array in ADF where the first node does not have key and starts with values which is not constant
I am trying to parse nested JSON in my pipeline to store the values such as table_name, lastSuccessfulWriteTimestamp, totalProcessedRecordsCount, dataFilesPath, schemaHistory_value1, schemaHistory_value2 from below JSON data using ADF dataflow activity.…
asked 2024-07-08T17:22:36.6666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(249.60000000000002, 5%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESH%3C/text%3E%3C/svg%3E)
Satish Hadapad
0
Reputation points
commented 2024-07-09T05:05:54.96+00:00
Smaran Thoomu
12,185
Reputation points Microsoft Vendor
1 answer
AZ DATA FACTORY - COPY Activity : 'Incorrect syntax near '>'
Hi, while i am trying to copy data using below dynamic value. SELECT * FROM @{pipeline().parameters.P_DS_Object_api} WHERE lastmodifieddate >= '@{formatDateTime(activity('watermarktable').output.firstRow.Watermarkvalue, 'yyyy-MM-dd HH:mm:ss')}' i am…
asked 2023-09-07T11:12:25.0733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(51.2, 64%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERB%3C/text%3E%3C/svg%3E)
Ram Babu
50
Reputation points
commented 2024-07-09T03:55:13.32+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 39%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Shweta Sapate
0
Reputation points