Content
Azure Synapse RESTAPI - Audience validation failed
Hi, I am trying to run a Azure synapse pipeline using RESTAPI. I have generated the bearer token in the first request and using the bearer token in the second request to run the pipeline. I am facing the below error: { …


Cannon update Synapse Triger via Rest API
Hi Team I am trying to create and update a synapse trigger via the REST API as described in this documentation link Creating a trigger works just perfectly, but updating an existing trigger does not happen. The request is received with status 203, but as…


Can't Create Database Scoped Credentials in Serverless synapse Database
When I run the below script, I am getting an error saying Incorrect syntax near IDENTITY


Database scope credential Not getting created in Azure Synapse studio for SQL Serverless Db pool
CREATE DATABASE SCOPED CREDENTIAL MyCosmosDbAccountCredential WITH IDENTITY = 'SHARED ACCESS SIGNATURE', SECRET = 's5zarR2pT0JWH9k8roipnWxUYBegOuFGjJpSjGlR36y86cW0GQ6RaaG8kGjsRAQoWMw1QKTkkX8HQtFpJjC8Hg=='; The above statement is not working…


Get the list of names via Get Metadata activity
Hi All, Below is the output of Get Metadata activity which contains name and type values for child items: Is it possible to just get the name values and stored within an array variable without using any iteration. Output =…


Azure_Synapses_parent-child-hierarchies
The need is to get the company hierarchy from employee and supervisors. Currently I'm able to do this directly in powerbi by using DAX functions: HierarchyPath = PATH(workforce[employee_number],workforce[supervisor_number]) The result is this: …


Install Python pycryptodome libraries for Apache Spark in Azure Synapse
Hi All, I am tying to install pycryptodome to azure synapse notebook. PFB details. scenario - I have created a notebook and Apache spark pool in azure synapse. I used the Below command to list the packages installed on pool. I don't see my required…


How many partitions is needed with 90 billions table row count to benefit the CCI?
Hi, Say for example I have a large fact hash-distributed (CCI) table partitioned monthly with 90 billions table row count (current): How many partition is needed to create to benefit from a clustered columnstore index? Based from question 1, is it…


How to execute Hive queries in Synapse spark
Hello! I am replacing a HDI cluster with Azure Synapse. My current HDI spark cluster executes some HIVE queries for data transformation. Is it possible to execute the same HIVE queries into Azure Synapse spark pool? Thanks, DR


How to Encrypt WEB API request and response using PyCryptodome
Recently I come up a scenario where I need to encrypt a WEB API request and response using PyCryptodome inside Synapse notebook activity. I am trying to make a call to LinkedIn API, but the request should be encrypted and similarly response should be…


Pagination of REST API call with "Link" in Header and rel="next"
Hello, I have a hard time doing pagination in ADF dataflow activity. Tried many of the below, nothing worked. My Link in Header is like below - Any idea how to work on it?


Copy Method in ADF - Polybase / Copy Command / Bulk Insert
Hi, We have data of million rows in delimited files placed in Azure Blob storage. We need to transfer to dedicated sql pool of Azure Synapse via Azure Data Factory So which copy method would be suitable in ADF - Copy command / polybase / Bulk Insert.…


Synapse - SparkPool - Error code AVAILABLE_WORKSPACE_CAPACITY_EXCEEDED.
Hi, I get this error that did not appear before, when i am making a simple query with little data. I have tried with a new pool but the error no disappear. AVAILABLE_WORKSPACE_CAPACITY_EXCEEDED: Livy session has failed. Session state: Error. Error code:…


Dynamic load balance
We have a Synapse Data Factory process that dynamically kicks off >200 extracts. As Data Factory has a 20 pipeline running concurrently limit, the remaining extracts are queued behind the initial 20 However, it appears to be evenly queuing the…


How to upgrade Spark version in Synaspe?
I would like to know how to upgrade Spark version in Synapse? I have a spark pool on spark 3.1 with a lot of notebooks and would like to upgrade to Spark 3.3 without having to create a new Spark pool and re-deploy my notebooks. I haven't found a…


Synapse Spark - AuthorizationPermissionMismatch
We have a storage account that uses CDM based files (D365 synapse link), we are trying to use the CDMUtil to create Spark based tables to enable us to reference these storage account files, however, we are getting the below error when running the…


Synapse Spark: Python logging to log file in Azure Data Lake Storage
I am working in Synapse Spark and building a logger function to handle error logging. I intend to push the logs to an existing log file (data.log) located in AzureDataLakeStorageAccount/Container/Folder/. In addition to the root logger I have added a…


Azure Synapse support for federated access
Hello, Is it possible to access SQL views created in one synapse workspace from another one? One potential option is to access it via the sql endpoint from a python code. However, is it possible to access the SQL serverless views of one workspace from…
We are trying to load the data in power bi through Synapse and we are facing this issue. Can some one help us with a solution.
OLE DB or ODBC error: [DataSource.Error] Microsoft SQL: 110806;A distributed query failed: Distribution(s):[2, 12-14, 21, 24-25, 40-41, 44, 58]The Microsoft Distributed Transaction Coordinator (MS DTC) has cancelled the distributed transaction.. An…


Is it possible to connect an Apache Spark Pool in an Azure Synapse workspace to an artifacts feed in Azure Devops?
I'm trying to configure an environment that my team are going to use with the Apache Spark pool that we've defined in our Synapse workspace. We have a number of internal packages that we have built and deployed to our own artefacts feed in Azure Devops…

