Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,402 questions
0 answers
Azure synapse batch job results to be returned from livy via API (without intermediate storage)
Is there a recommended way to return a small amount of output from a batch job that is executed via livy? In the best case I would return all of stdout from the driver. Otherwise it might be enough to return a short string that says something…
asked 2022-09-26T23:54:34.27+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 78%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDB%3C/text%3E%3C/svg%3E)
David Beavon
976
Reputation points
commented 2022-10-11T18:53:52.607+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 26%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HimanshuSinha-msft
19,381
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Azure log analytics: How to track SQL usage?
I have enabled log analytics for synapse workspace. My next goal is to use log analytics to write Kusto query to find which tables are queried the most and which users are running the most queries. Using this link I see all the tables that are…
asked 2022-10-06T14:25:04.163+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 20%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECP%3C/text%3E%3C/svg%3E)
Charmin Patel
21
Reputation points
commented 2022-10-11T18:49:31.19+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
BhargavaGunnam-MSFT
26,316
Reputation points Microsoft Employee
0 answers
Azure Pipeline Copy Rest Api to storage returning only 1 row
I have a pipeline built in azure synapse pipeline - that reads data from a rest API source and a target sink as blob storage. the pipeline was working fine with and without pagination till now. But since a while it has been returning only 1 row…
asked 2022-09-13T15:23:35.317+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 92%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVM%3C/text%3E%3C/svg%3E)
Vamsi Madhav
46
Reputation points
commented 2022-10-11T18:37:30.437+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 70%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
MartinJaffer-MSFT
26,031
Reputation points
3 answers
DR for Azure SQL DB/Synapse
Hi All, We have enabled LTR backup for Azure SQL DB's and Geo Backup policy is by default enabled in Synapse. Below are my queries: Does the LTR backup for Azure SQL DB's and Geo Backup in Synapse are created in a regional pair or in a Geo…
asked 2022-10-06T14:41:50.327+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 27%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPJ%3C/text%3E%3C/svg%3E)
Priya Jha
866
Reputation points
commented 2022-10-11T18:31:24.193+00:00
GeethaThatipatri-MSFT
27,717
Reputation points Microsoft Employee
0 answers
Unix time stamp conversion in Azure synapse analytics
Hello Team, I am using the below script to do refining the data in silver layer: For particular column CreatedDate the data is appearing in the Unix format.Please refer the below : I need to convert the data into YYYY-MM-DD :…
asked 2022-10-10T13:36:47.647+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 13%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Rohit Kulkarni
676
Reputation points
commented 2022-10-11T14:50:29.253+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 32%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMT%3C/text%3E%3C/svg%3E)
Michael Taylor
48,656
Reputation points
1 answer
One of the answers was accepted by the question author.
Adding whl files to an Azure Synapse spark pool
According to the documentation, we should be able to add custom libraries as follows: az synapse spark pool update --name testpool \ --workspace-name testsynapseworkspace --resource-group rg \ --package-action Add --package package1.jar…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 88%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV%3C/text%3E%3C/svg%3E)
2 answers
What is the best way to connect to Serverless SQL from a Synapse pySpark Notebook?
I am currently trying to use the documentation for AD Based authentication at…
asked 2022-07-28T20:27:08.677+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 68%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ET%3C/text%3E%3C/svg%3E)
Tim
31
Reputation points
commented 2022-10-11T14:25:08.887+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 3%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBD%3C/text%3E%3C/svg%3E)
BLIN-DESDOITS Jean
1
Reputation point
1 answer
PublicNetworkAccessDenied
hello, i want to use Private endpoints in my virtual network to access synapse what i must do? it is give me this error when i try to linked service with source from synapse
asked 2022-09-28T15:51:18.883+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 8%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anonymous
commented 2022-10-10T23:14:02.073+00:00
BhargavaGunnam-MSFT
26,316
Reputation points Microsoft Employee
1 answer
Query timestamp_micros value of parquet file using Azure Synapse
I have a parquet file with one of the timestamp fields with following parquet schema, uploaded to Azure Synapse workspace { file_name: 'changesschema.parquet', name: 'ctime', type: 'INT64', type_length: '0', repetition_type:…
asked 2022-10-04T05:50:51.96+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 60%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
srigowri
26
Reputation points
commented 2022-10-10T22:47:37.51+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 9%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
KranthiPakala-MSFT
46,422
Reputation points Microsoft Employee
1 answer
temporary vs non-temporary staging sql dedicated pool tables
I am ingesting data from Azure SQL DB to Synapse DWH and I have seen people are following 2 approaches: 1) Azure SQL DB --> Data Lake -----------------------------> temporary staging SQL dedicated pool table --> production SQL dedicated pool…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 54%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EK%3C/text%3E%3C/svg%3E)
0 answers
Data lineage reporting to purview timedout.
Hi I was having a problem with Synapse copy data reporting lineage to purview. So i was doing a POC on Synapse and Purview but apparently lineage isn't being reported as seen on the screenshot even though it is connected on purview. Copy is from…
asked 2022-09-23T01:54:20.287+00:00
Jan Clint Viernes
1
Reputation point
commented 2022-10-10T04:11:07.383+00:00
PRADEEPCHEEKATLA-MSFT
77,901
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Capture Event Hub without duplication
What is the best way to capture event hubs without duplication. Since event hubs delivery at least once. We can have the same event multiple times. As you can imagine I don't want to store the same event hub twice. I just want to store each event hub…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 27%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDS%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
How to Dynamically pass the Filename in Azure Pipeline in Synapse analytics??
Hello Everyone, I have written a code in synapse notebook using pyspark for transformation. I have a task in which i have to dynamically change the file name when we will deploy this code in our pipeline. In Screenshot file_name is the path…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(44.800000000000004, 15%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
1 answer
Backup difference comparison
Hi All, I have a below scenario wherein i accidently dropped 5 tables from my Azure SQL DB and Azure SQL Data warehouse. I do have LTR backup for Azure SQL DB and restore point for SQL Data warehouse. Is it possible to restore just the dropped…
asked 2022-10-07T14:54:21.767+00:00
Priya Jha
866
Reputation points
answered 2022-10-07T15:08:22.6+00:00
Bjoern Peters
8,781
Reputation points
1 answer
dacpac creation from Azure Synapse dedicated SQL Pool using Azure DevOps
Hi, I have created the sql dacpac file from azure SQL database using Devops pipeline inbuilt task 'Azure SQL database deployment'. Now I am trying to create dacpac file from azure synapse SQL Pool. but I didn't find any related task or PowerShell…
asked 2022-10-03T19:19:04.903+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 0%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESG%3C/text%3E%3C/svg%3E)
sachin gupta
376
Reputation points
answered 2022-10-07T14:58:25.897+00:00
Nandan Hegde
29,896
Reputation points MVP
1 answer
migrating SAP to Azure synapse
What are the use cases for migrating SAP to Azure synapse? Tks
asked 2022-10-01T17:19:53.46+00:00
Anonymous
commented 2022-10-07T09:03:21.713+00:00
AnnuKumari-MSFT
31,146
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
How to create Azure Synapse Link in multi-tenant architecture between directories?
I am trying to create a Azure Synapse Link to connect Microsoft Dataverse data to Azure Synapse Analytics. I am following this documentation https://learn.microsoft.com/en-us/power-apps/maker/data-platform/azure-synapse-link-synapse. One of the…
asked 2022-09-30T11:13:33.87+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 81%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKU%3C/text%3E%3C/svg%3E)
Klaus Uhle
62
Reputation points
accepted 2022-10-07T08:51:17.053+00:00
Klaus Uhle
62
Reputation points
2 answers
One of the answers was accepted by the question author.
Error when transferring data from Azure Synapse Serverless pool to Azure SQL DB
I am trying to copy data from tables present in Azure Synapse Serverless pool to Azure SQL DB using copy activity of ADF and receiving this below error intermittently for any table: Failure happened on 'Source' side.…
asked 2022-08-26T14:00:09.483+00:00
Priya Jha
866
Reputation points
commented 2022-10-07T06:43:42.043+00:00
AnnuKumari-MSFT
31,146
Reputation points Microsoft Employee
1 answer
Source to Sink (Excel to Parquet) - File Name issue
Hi, I have a data flow where my source is an excel file and sink is parquet file. I have multiple excel files in source folder. And I want to copy from excelt o parquet. The filenames should be of same name as in source. For example if I have…
asked 2022-10-05T17:50:33.347+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 51%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESD%3C/text%3E%3C/svg%3E)
Santhi Dhanuskodi
66
Reputation points
commented 2022-10-07T05:45:01.73+00:00
ShaikMaheer-MSFT
37,896
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Git Configuration Disconnected - Azure
Hi, Git Configuration is disconnected in Azure Synapse SQL. I would like to know who has disconnected that. Is there a way to track this? I cannt find this in 'Activity Log'
asked 2022-10-05T07:37:18.263+00:00
Santhi Dhanuskodi
66
Reputation points
commented 2022-10-07T05:33:27.167+00:00
Pratik Somaiya
4,201
Reputation points