Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,553 questions
1 answer
Querying table index stats not working
I am trying to query my tables index fragmentation in Azure Synapse studio with query: SELECT IPS.Index_type_desc, IPS.avg_fragmentation_in_percent, IPS.avg_fragment_size_in_pages, IPS.avg_page_space_used_in_percent, …
asked 2024-06-14T09:17:01.14+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 7.000000000000001%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJH%3C/text%3E%3C/svg%3E)
Jani Hämäläinen
20
Reputation points
answered 2024-06-14T21:25:10.3566667+00:00
Erland Sommarskog
103.5K
Reputation points MVP
1 answer
IP's to whitelist for Integration Runtime to access 3rd party Sql server
Trying to connect Azure Data Factory to a 3rd party sql server. What IP's will the 3rd party need to whitelist for Inegration Runtime to access their Sql Server. In the US East Region.
asked 2024-06-14T20:20:00.3166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 50%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJM%3C/text%3E%3C/svg%3E)
Jim Murphy
0
Reputation points
answered 2024-06-14T20:47:47.4633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 24%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAA%3C/text%3E%3C/svg%3E)
akinbade abiola
4,670
Reputation points
2 answers
Extract Date from FileName in Copy Activity (ADF)
We have multiple files with names as 'Amazon US Daily Sales Diagnostic_2021-05-10.xlsx', we want a solution to fetch this date from the filename and add a column by name 'date' in the table with this value through ADF by using copy activity.
asked 2021-05-29T13:35:11.883+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 50%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAC%3C/text%3E%3C/svg%3E)
Ayush Chauhan
116
Reputation points
commented 2024-06-14T17:43:31.43+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(9.6, 57.99999999999999%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ET%3C/text%3E%3C/svg%3E)
Thyagaraj
0
Reputation points
0 answers
Data flow is not sinking all data because I have 84 json files in different sub-folders (of which 78 files are blank, and 6 files have data). But when I delete the 78 blank files (and remain with 6 files with data), the data flow will sink all results.
Hello. Please support, my data flow pipeline is not sinking all data. This happens whenever I have 84 json files each in different sub-folders (of which 78 files are blank, and 6 files have data). But when I delete the 78 blank json files (and only…
asked 2024-06-14T17:35:51.8166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 6%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Babu Musanga
0
Reputation points
asked 2024-06-14T17:35:51.8166667+00:00
Babu Musanga
0
Reputation points
2 answers
One of the answers was accepted by the question author.
Azure Data Factory - DF-REST_001 - Error response from server: Some(), Status code: 404. Please check your request url and body
I setup a pipeline and dataflow in Azure Datafactory. The purpose is get a token and then get data from an api. I am getting the token fine. then I set a token variable and pass that token into the data flow. I'm getting an error in the source data…
asked 2023-12-12T15:23:35.23+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 42%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Asha Richardson
26
Reputation points
commented 2024-06-14T17:05:15.0266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 17%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGM%3C/text%3E%3C/svg%3E)
Gary Melhaff - US
6
Reputation points
0 answers
Azure Data Factory - same CSV/dataset, one data flow works, one not
I have a csv file and a correspondig data set. Data Flow 1: is processing the file without any issue Data Flow 2: is failing Both source items are configured identically. Route cause of the failing is, that in the csv file, one colum is producing line…
0 answers
azur et CDN
Hello, we have an AZURE account with encoded videos stored on Azure CDN. It's just the Azure encoding service which closes on June 30. Our data stored on Microsoft CDN remains online and accessible?
asked 2024-06-14T16:14:43.7666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 6%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHI%3C/text%3E%3C/svg%3E)
HDP INTERACTIVE
0
Reputation points
asked 2024-06-14T16:14:43.7666667+00:00
HDP INTERACTIVE
0
Reputation points
0 answers
Add new row to text file in blob storage via data factory
I have a data factory pipeline, and I would like to add an activity where I want to take a .txt file stored in blob storage and append a new row to it. The row should include the current datetime as well as a variable obtained in a previous…
asked 2024-06-14T15:14:06.1533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 68%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELK%3C/text%3E%3C/svg%3E)
Louise Kirkham
0
Reputation points
asked 2024-06-14T15:14:06.1533333+00:00
Louise Kirkham
0
Reputation points
1 answer
Azure Data Factory - mark source rows as "processed"
I wan't to read unprocessed data from asource table and write it to a target table. Normally, in other ETL Tools, I would add a "processed" flag column to sourceTable update it after successfully insert into targetTable with…
asked 2024-06-14T10:41:55.6066667+00:00
Jörg Lang
60
Reputation points
answered 2024-06-14T14:49:21.45+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 4%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Sina Salam
5,471
Reputation points
0 answers
Configure ADF with Dev Github
hi can you tell me the steps/process to configure dev ADF github so I can proceed to create ADF pipeline. thanks Sudha
asked 2024-06-14T12:54:41.05+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 57.00000000000001%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
Sudha Achuthan
0
Reputation points
asked 2024-06-14T12:54:41.05+00:00
Sudha Achuthan
0
Reputation points
2 answers
convert excel column headers as row headers in azure data factory
need to convert below example input from excel source - id qn1 qn2 qn3 ................ max q50 1 qr1 qr2 qr3 ................max qr50 2 qr1 qr2 qr3 ................max qr50 desired output - id qn qr 1 qn1 qr1 1 qn2 qr2 1 qn3 qr3 . . 1 qn50…
asked 2024-06-13T04:34:41.5066667+00:00
Sudha Achuthan
0
Reputation points
commented 2024-06-14T12:51:02.8466667+00:00
Sudha Achuthan
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Azure Data Factory - Data Flow - parameters in group by
Hi, I want to use parameter in Azure Data Factory Mapping Data Flows to determine which columns will by used to group by in Aggregation tool. I used Array as parameter type. I used this parameter in group by in aggregate tool. However…
asked 2020-12-17T10:31:02.933+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 69%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGA%3C/text%3E%3C/svg%3E)
Gabryel, Andrzej
166
Reputation points
commented 2024-06-14T12:29:28.11+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 80%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAN%3C/text%3E%3C/svg%3E)
Anil Narayan Gawli
0
Reputation points
1 answer
i want to retrieve pipeline details, including the repository and branch name, using an API call from Data Factory.
I am trying to fetch pipeline data from Data Factory using an API call. I am able to retrieve the pipeline data, but I get data for all the pipelines in ADF. Now, I want to fetch the pipeline data by branch. I have checked almost all the APIs in the…
asked 2024-06-13T05:41:25+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 37%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPP%3C/text%3E%3C/svg%3E)
Parthkumar Pavra
0
Reputation points
edited the question 2024-06-14T12:29:17.0466667+00:00
Ryan Hill
26,771
Reputation points Microsoft Employee
0 answers
Is it possible to run an SSIS package with FUZZYGROUPING transformation in Azure?
I am using Fuzzy grouping transformation in SSIS locally "on premises", but I would like to explore the option to run it in Azure…
asked 2024-06-13T09:35:48.62+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(179.20000000000002, 27%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJM%3C/text%3E%3C/svg%3E)
Jose M Naranjo-Espinosa
0
Reputation points
commented 2024-06-14T11:33:56.63+00:00
PRADEEPCHEEKATLA-MSFT
82,191
Reputation points Microsoft Employee
1 answer
How to store sharepoint list columns in an excel using Azure Data Factory
We have a sharepoint list that has more than 50 fields and lots of rows. Currently I'm using Web activity where I'm passing /sites/site-id/lists/list-id/items? expand=fields. Thr first field is title, second is organization, etc. But in the output I'm…
asked 2024-06-13T04:29:56.2666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 35%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMY%3C/text%3E%3C/svg%3E)
Mansi Yadav
40
Reputation points
edited a comment 2024-06-14T09:28:21.62+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
7,345
Reputation points Microsoft Vendor
1 answer
How to transform sharepoint list items into an excel format
I'm trying to fetch the columns inside a sharepoint list using below MS Graph API:- sites/site-id/lists/list-id/items?expand=fields Below is the output that I'm getting:- { "@odata.context":…
asked 2024-06-13T16:16:56.5766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 79%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
Rayne
40
Reputation points
answered 2024-06-14T09:05:50.8366667+00:00
Amira Bedhiafi
17,786
Reputation points
1 answer
How to retrieve both debug and triggered pipeline run details for ADF using a single REST API call using ADF web activity
How can I obtain the latest run date of a pipeline (both Triggered and Debug run details) utilizing just the pipeline name? The first URL retrieves the triggered pipeline details with the ADF web activity, and the second URL retrieves the debug pipeline…
asked 2024-06-10T08:51:06.91+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 73%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECP%3C/text%3E%3C/svg%3E)
Cherukuri, Praveen Kumar
0
Reputation points
commented 2024-06-14T09:02:56.67+00:00
phemanth
7,345
Reputation points Microsoft Vendor
1 answer
Azure Data Factory read spreadsheet with Protected View
I have a logic app workflow that is triggered when a new email is received in a mailbox. The workflow copies the attachment (spreadsheet) in the email, to Azure Blob Storage. I then trigger an Azure Data Factory pipeline to read the contents of the…
asked 2024-06-13T19:34:11.5233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 4%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENM%3C/text%3E%3C/svg%3E)
Naveed Mohammad
0
Reputation points
answered 2024-06-14T09:02:42.4733333+00:00
Amira Bedhiafi
17,786
Reputation points
0 answers
Get jsonlines response from Rest API and load into SQL DB using ADF pipeline
Hi Guys, Requirement is to call Rest APIs which will response data in jsonlines format (not Json array) and load data into table. Tried with Web activity and Set Variable to capture the response but not getting the response in expected format. Can you…
asked 2024-04-15T10:19:39.51+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 45%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
SarmisthaS
0
Reputation points
commented 2024-06-14T08:45:01.2766667+00:00
phemanth
7,345
Reputation points Microsoft Vendor
0 answers
Synapse Dedicated pool index stats
I try to get the fragmentation of my indexes with code SELECT IPS.Index_type_desc, IPS.avg_fragmentation_in_percent, IPS.avg_fragment_size_in_pages, IPS.avg_page_space_used_in_percent, IPS.record_count, …
asked 2024-06-14T08:39:27.7+00:00
Jani Hämäläinen
20
Reputation points
asked 2024-06-14T08:39:27.7+00:00
Jani Hämäläinen
20
Reputation points