Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,687 questions
1 answer
Schedule a pipeline in Azure Synapse analytics
How can I schedule a trigger pipeline in Azure Synapse analytics to run every 30 minutes, but avoid starting when another pipeline is running or at certain specified times (such as between 1 am to 3 am)? I am seeking guidance on the best approach for…
asked 2024-07-24T07:33:29.9833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 69%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Aditya Singh
105
Reputation points
commented 2024-07-25T05:19:18.91+00:00
Aditya Singh
105
Reputation points
1 answer
Upsert Data in SQL Server Using Synapse Notebook and Pyspark
How do I use pyspark in a synapse notebook to upsert data in SQL Server? I am able to read the table with the following code: df = spark.read.jdbc(url=jdbc_url, table="Dim.table_name", properties=properties) But I am not sure how to upsert…
asked 2024-07-17T12:14:43.41+00:00
Aditya Singh
105
Reputation points
commented 2024-07-25T05:16:09.67+00:00
PRADEEPCHEEKATLA-MSFT
85,511
Reputation points Microsoft Employee
2 answers
Restore KekIdentity in Synapse Workspace
Hi, We deleted the managed identity for our synapse workspace that was the KekIdentity And now, when we open the workspace, none of the resources will load. I don't see any documentation on how to restore the KekIdentity for the Azure…
asked 2024-07-12T22:43:20.23+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 40%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDC%3C/text%3E%3C/svg%3E)
Dance, Cody R. (ALT)
25
Reputation points
commented 2024-07-25T05:13:57.15+00:00
PRADEEPCHEEKATLA-MSFT
85,511
Reputation points Microsoft Employee
0 answers
Missing "smallrc" DB role from Synapse Analytics dedicated SQL pools
Hi All, I am a data engineer. Since yesterday we have observed that the "smallrc" DB role is missing from all synapse dedicated sql pools in all our environments (dev/stage/prod). All our synapse instances are provisioned in the AU East Azure…
asked 2024-07-22T06:50:25.8133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 98%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAN%3C/text%3E%3C/svg%3E)
Ashish Naik
0
Reputation points
commented 2024-07-25T05:01:36.89+00:00
PRADEEPCHEEKATLA-MSFT
85,511
Reputation points Microsoft Employee
2 answers
Facing issues to connect with synapse workspace/azure sql db via oauth
I want to connect to and access my Synapse Workspace and SQL Pool using OAuth. For this purpose, I have created an app in Azure and assigned the following API permissions: Microsoft Graph: Directory.Read.All (Delegated) Directory.ReadWrite.All…
asked 2024-07-15T05:04:01.89+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 23%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
SUBHAM KUMAR
0
Reputation points
commented 2024-07-25T04:53:27.1466667+00:00
SUBHAM KUMAR
0
Reputation points
1 answer
One of the answers was accepted by the question author.
transformation changes in silver and bronze layer
Hi , what transformation takes place between silver and gold layer .i.e have loaded data in bromze layer whichn is my bronze layer and i transformed data here ... then what will happend to silver to gold layer apart from pk,fk joins and all
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
1 answer
how to update Azure Synapse spark pool with an exiting configuration in workspace
I have an existing Synapse configuration that has been set from the Synapse studio. Now I have a new spark pool created and want to bind that configuration to the pool. On the portal, it can be set on the sparkpool. But what is the programmatic way to do…
asked 2024-07-24T15:00:40.3766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 80%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EB%3C/text%3E%3C/svg%3E)
BenZ
0
Reputation points
answered 2024-07-24T23:40:27.64+00:00
Amira Bedhiafi
19,946
Reputation points
1 answer
Azure Synapse Analytics Serverless - cross database queries
Synapse Analytics Serverless. Scenario: a ViewB in DatabaseB points to ViewA in DatabaseA where same user is created on both databases on order to read ViewB data Question is how we can hide user access on DatabaseA so only ViewB is reachable but…
asked 2024-07-24T17:19:44.54+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 28.999999999999996%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMP%3C/text%3E%3C/svg%3E)
Marcelo Petit
0
Reputation points
answered 2024-07-24T23:23:09.2233333+00:00
Amira Bedhiafi
19,946
Reputation points
2 answers
Why is FailureWebHook not executed in this case?
Please consider the following pipeline. The Execute Pipeline activity failed, so the ProcessTMONData notebook was skipped. Why was the failure webhook not triggered, as it's supposed to trigger when EITHER: ForEachEndpoint fails ProcessTMONData is…
asked 2024-07-23T12:10:36.8466667+00:00
Psychotechnopath
31
Reputation points
edited a comment 2024-07-24T23:18:18.8833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Bhargava-MSFT
29,036
Reputation points Microsoft Employee
0 answers
Datetime2 error when querying parquet file produced from Synapse copy activity from OData Business Central Source (data seems to be changed from 0001-01-01 to 0000-12-30)
I have a pipeline running a copy activity to copy data from Business Central OData source to parquet file everyday. Everything works fine before 16/07/2024 but problem arises starting from 17/07/2024. The parquet file no longer be able to query from…
asked 2024-07-24T15:13:17.94+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(233.6, 10%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVC%3C/text%3E%3C/svg%3E)
Victor Chu
0
Reputation points
commented 2024-07-24T20:28:25.9433333+00:00
Bhargava-MSFT
29,036
Reputation points Microsoft Employee
0 answers
Azure Synapse "Message Type Not Found" error when running stored procedure: Appears when lines of code reach certain line length
Hi, We need some technical guidance from either a Azure Synapse specialist to resolve an issue which we are unable to understand / fix despite running through the sql logic bit-by-bit, line-by-line. Any guidance or help would be much appreciated. We have…
asked 2024-07-22T17:24:40.0233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 76%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMK%3C/text%3E%3C/svg%3E)
Michael King
0
Reputation points
commented 2024-07-24T17:34:49.14+00:00
Michael King
0
Reputation points
1 answer
Azure Synapse Analytics Data Flow: How do the Table Actions and Update Methods work & can you conditionally Truncate with 1 Data Flow?
I am working in Azure Synapse Analytics and making a dataflow, but I'm struggling with understanding some things. (I'm using Azure Data Lake Storage Gen 2 and Delta file for my sink) For the sink activity: what is the difference between the update…
asked 2024-07-18T21:18:12.84+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 94%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
Joshua
0
Reputation points
commented 2024-07-24T14:47:17.6333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
400
Reputation points Microsoft Vendor
0 answers
Getting error like 'FAILED CLUSTER CREATION' for apache spark pool compute cluster
Apache spark pool compute cluster is not running since 20th July. error show sin 'LivySessionCreationTimeoutException: Livy session creation timed out.' has anybody else faced this issue? I have created a new cluster and tried but that's showing the same…
asked 2024-07-22T18:56:02.6266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 32%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGP%3C/text%3E%3C/svg%3E)
Ghosh, Paramita
5
Reputation points
commented 2024-07-24T14:46:15.93+00:00
Chandra Boorla
400
Reputation points Microsoft Vendor
1 answer
Error due to cloned Pipeline after it was renamed and has been published
Hi all I cloned a pipeline and then renamed it and published the changes: So Now I have FolderLive PipelineLIVE PipelineTEST FolderTEST Perfect! so all published and ready for the next step of moving PipelineTEST in to FolderTEST and publish. …
asked 2024-07-24T09:02:18.85+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 89%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGJ%3C/text%3E%3C/svg%3E)
Greenwood, Justin
0
Reputation points
answered 2024-07-24T14:39:21.2+00:00
Chandra Boorla
400
Reputation points Microsoft Vendor
1 answer
Data is missing from management API using service principle. ex:resources
I have a requirement to load datasets from management.Azure.com (Management API), for example: Resources, Assessments, Tasks, and Secure Score. I have created a service principal in app registration and have been able to obtain the access token. However,…
asked 2024-07-11T22:11:50.9166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 35%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMR%3C/text%3E%3C/svg%3E)
Mahesh Reddy
0
Reputation points
commented 2024-07-24T14:25:32.3466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
12,615
Reputation points Microsoft Vendor
1 answer
Synapse Link for Dataverse-SYN-806: Default storage container in Synapse workspace is not accessible error while enbling tables
Hi, I'm trying to setup a synapse link via dataverse to F&O. I'm able to setup the connection but when I enable tables sync status is throwing out error . Error details - SYN-806: Default storage container in Synapse workspace is not…
asked 2024-05-17T12:32:44.3633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 92%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Sandz
66
Reputation points
commented 2024-07-24T13:19:20.2733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 78%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGM%3C/text%3E%3C/svg%3E)
Grossmann Manuel
0
Reputation points
0 answers
Synapse data warehouse and use
Hi friends, can we build a data mart kind of architecture for multiple domains in Synapse? How to built data marts for specific business entities in Synapse?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 15%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
4 answers
How to make manual failover to Azure Gen2 namespace enabled for disaster recovery plan
As a disaster recovery drill test for Azure adls namespace enabled accounts . I could see failover option for blob storage and but for adls namespace enabled accounts the failover option is disabled. Can anyone help in know how to do failover for Azure…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(124.80000000000001, 85%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EK%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
1 answer
Problem with Power BI
Hi I am having a problem linking a button with a bookmark. When I click on the button nothing happens. I have referenced chatgpt and I can't get a straight answer. Thank you
asked 2024-07-24T04:40:03.14+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 31%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGH%3C/text%3E%3C/svg%3E)
Geoff Hughes
0
Reputation points
answered 2024-07-24T06:13:35.91+00:00
PRADEEPCHEEKATLA-MSFT
85,511
Reputation points Microsoft Employee
0 answers
Connect to eventhub using Managed service identity (MSI) from Azure synapse analytics
I am trying to write the data to azure eventhubs from the azure synapse analytics notebook in scala using Managed service identity (MSI). Can you please help me with code for that. Note that i was able to successfully connect using the SAS key.
asked 2024-07-24T00:29:21.13+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(182.40000000000003, 94%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDK%3C/text%3E%3C/svg%3E)
Dheeraj Kumar Reddy Jogi
21
Reputation points Microsoft Employee
asked 2024-07-24T00:29:21.13+00:00
Dheeraj Kumar Reddy Jogi
21
Reputation points Microsoft Employee