Microsoft Q&A

Azure Synapse Analytics

2,621 questions

An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.

Browse all Azure tags

2,621 questions with Azure Synapse Analytics tags

Sort by: Updated
1 answer One of the answers was accepted by the question author.

Azure Synapse RESTAPI - Audience validation failed

Hi, I am trying to run a Azure synapse pipeline using RESTAPI. I have generated the bearer token in the first request and using the bearer token in the second request to run the pipeline. I am facing the below error: { …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2021-04-10T13:00:33.287+00:00
Sangilimurugan S 46 Reputation points
commented 2023-02-03T11:08:31.5766667+00:00
Nisha Singh 1 Reputation point
2 answers

Cannon update Synapse Triger via Rest API

Hi Team I am trying to create and update a synapse trigger via the REST API as described in this documentation link Creating a trigger works just perfectly, but updating an existing trigger does not happen. The request is received with status 203, but as…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2023-02-02T13:00:31.1433333+00:00
Viktor Bakayov 0 Reputation points
answered 2023-02-03T11:02:08.66+00:00
Viktor Bakayov 0 Reputation points
2 answers One of the answers was accepted by the question author.

Can't Create Database Scoped Credentials in Serverless synapse Database

When I run the below script, I am getting an error saying Incorrect syntax near IDENTITY

Azure SQL Database
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2022-08-15T22:57:56.153+00:00
Teja.Gudipati 26 Reputation points
answered 2023-02-03T10:59:54.74+00:00
Agneum 21 Reputation points
1 answer

Database scope credential Not getting created in Azure Synapse studio for SQL Serverless Db pool

CREATE DATABASE SCOPED CREDENTIAL MyCosmosDbAccountCredential WITH IDENTITY = 'SHARED ACCESS SIGNATURE', SECRET = 's5zarR2pT0JWH9k8roipnWxUYBegOuFGjJpSjGlR36y86cW0GQ6RaaG8kGjsRAQoWMw1QKTkkX8HQtFpJjC8Hg=='; The above statement is not working…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2021-11-09T13:23:25.63+00:00
Sahil Bansal 1 Reputation point
commented 2023-02-03T10:51:40.2633333+00:00
Agneum 21 Reputation points
1 answer

Get the list of names via Get Metadata activity

Hi All, Below is the output of Get Metadata activity which contains name and type values for child items: Is it possible to just get the name values and stored within an array variable without using any iteration. Output =…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,298 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2023-02-02T12:20:22.89+00:00
Priya Jha 721 Reputation points
answered 2023-02-03T10:16:14.89+00:00
AnnuKumari-MSFT 19,116 Reputation points Microsoft Employee
3 answers

Azure_Synapses_parent-child-hierarchies

The need is to get the company hierarchy from employee and supervisors. Currently I'm able to do this directly in powerbi by using DAX functions: HierarchyPath = PATH(workforce[employee_number],workforce[supervisor_number]) The result is this: …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2023-01-06T15:04:32.02+00:00
Daniel Moreira 1 Reputation point
commented 2023-02-03T09:04:18.8766667+00:00
Daniel Moreira 1 Reputation point
1 answer

Install Python pycryptodome libraries for Apache Spark in Azure Synapse

Hi All, I am tying to install pycryptodome to azure synapse notebook. PFB details. scenario - I have created a notebook and Apache spark pool in azure synapse. I used the Below command to list the packages installed on pool. I don't see my required…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2023-02-01T00:59:23.05+00:00
sachin gupta 341 Reputation points
answered 2023-02-03T08:35:41.8+00:00
PRADEEPCHEEKATLA-MSFT 53,276 Reputation points Microsoft Employee
1 answer

How many partitions is needed with 90 billions table row count to benefit the CCI?

Hi, Say for example I have a large fact hash-distributed (CCI) table partitioned monthly with 90 billions table row count (current): How many partition is needed to create to benefit from a clustered columnstore index? Based from question 1, is it…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
Azure SQL Database
asked 2023-01-30T15:30:29.5666667+00:00
Anthony Marasigan 21 Reputation points
commented 2023-02-03T07:33:17.3966667+00:00
Anthony Marasigan 21 Reputation points
1 answer

How to execute Hive queries in Synapse spark

Hello! I am replacing a HDI cluster with Azure Synapse. My current HDI spark cluster executes some HIVE queries for data transformation. Is it possible to execute the same HIVE queries into Azure Synapse spark pool? Thanks, DR

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
164 questions
asked 2023-02-03T02:56:53.8133333+00:00
Dharmesh Rathod 0 Reputation points
edited an answer 2023-02-03T06:10:57.43+00:00
PRADEEPCHEEKATLA-MSFT 53,276 Reputation points Microsoft Employee
1 answer

How to Encrypt WEB API request and response using PyCryptodome

Recently I come up a scenario where I need to encrypt a WEB API request and response using PyCryptodome inside Synapse notebook activity. I am trying to make a call to LinkedIn API, but the request should be encrypted and similarly response should be…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2023-02-03T02:08:26.36+00:00
sachin gupta 341 Reputation points
answered 2023-02-03T04:09:56.9666667+00:00
Aido 0 Reputation points
1 answer

Pagination of REST API call with "Link" in Header and rel="next"

Hello, I have a hard time doing pagination in ADF dataflow activity. Tried many of the below, nothing worked. My Link in Header is like below - Any idea how to work on it?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,298 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2023-02-02T01:11:52.59+00:00
Kumar, Arun 21 Reputation points
answered 2023-02-03T00:30:15.25+00:00
Kumar, Arun 21 Reputation points
1 answer

Copy Method in ADF - Polybase / Copy Command / Bulk Insert

Hi, We have data of million rows in delimited files placed in Azure Blob storage. We need to transfer to dedicated sql pool of Azure Synapse via Azure Data Factory So which copy method would be suitable in ADF - Copy command / polybase / Bulk Insert.…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,298 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2023-02-02T07:19:52.46+00:00
Murugeswari 441 Reputation points
answered 2023-02-03T00:02:22.0966667+00:00
KranthiPakala-MSFT 33,136 Reputation points Microsoft Employee
2 answers

Synapse - SparkPool - Error code AVAILABLE_WORKSPACE_CAPACITY_EXCEEDED.

Hi, I get this error that did not appear before, when i am making a simple query with little data. I have tried with a new pool but the error no disappear. AVAILABLE_WORKSPACE_CAPACITY_EXCEEDED: Livy session has failed. Session state: Error. Error code:…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2023-01-31T08:43:43.39+00:00
Marielena Chavez 0 Reputation points
answered 2023-02-02T22:18:50.12+00:00
BhargavaGunnam-MSFT 7,411 Reputation points Microsoft Employee
0 answers

Dynamic load balance

We have a Synapse Data Factory process that dynamically kicks off >200 extracts. As Data Factory has a 20 pipeline running concurrently limit, the remaining extracts are queued behind the initial 20 However, it appears to be evenly queuing the…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,298 questions
asked 2023-01-26T23:05:01.87+00:00
Ryan Abbey 1,061 Reputation points
commented 2023-02-02T21:56:52.1133333+00:00
BhargavaGunnam-MSFT 7,411 Reputation points Microsoft Employee
1 answer

How to upgrade Spark version in Synaspe?

I would like to know how to upgrade Spark version in Synapse? I have a spark pool on spark 3.1 with a lot of notebooks and would like to upgrade to Spark 3.3 without having to create a new Spark pool and re-deploy my notebooks. I haven't found a…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2023-01-30T08:37:49.1866667+00:00
Pierre Gourven 21 Reputation points
answered 2023-02-02T21:44:30.93+00:00
BhargavaGunnam-MSFT 7,411 Reputation points Microsoft Employee
0 answers

Synapse Spark - AuthorizationPermissionMismatch

We have a storage account that uses CDM based files (D365 synapse link), we are trying to use the CDMUtil to create Spark based tables to enable us to reference these storage account files, however, we are getting the below error when running the…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2023-01-31T21:01:33.18+00:00
Ryan Abbey 1,061 Reputation points
commented 2023-02-02T20:20:57.32+00:00
Ryan Abbey 1,061 Reputation points
0 answers

Synapse Spark: Python logging to log file in Azure Data Lake Storage

I am working in Synapse Spark and building a logger function to handle error logging. I intend to push the logs to an existing log file (data.log) located in AzureDataLakeStorageAccount/Container/Folder/. In addition to the root logger I have added a…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
896 questions
asked 2023-02-01T04:35:10.99+00:00
Sumit Bhatnagar 1 Reputation point
commented 2023-02-02T17:37:37.7066667+00:00
MartinJaffer-MSFT 24,091 Reputation points Microsoft Employee
0 answers

Azure Synapse support for federated access

Hello, Is it possible to access SQL views created in one synapse workspace from another one? One potential option is to access it via the sql endpoint from a python code. However, is it possible to access the SQL serverless views of one workspace from…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2023-02-02T17:00:47.24+00:00
saurabh 206 Reputation points
2 answers

We are trying to load the data in power bi through Synapse and we are facing this issue. Can some one help us with a solution.

OLE DB or ODBC error: [DataSource.Error] Microsoft SQL: 110806;A distributed query failed: Distribution(s):[2, 12-14, 21, 24-25, 40-41, 44, 58]The Microsoft Distributed Transaction Coordinator (MS DTC) has cancelled the distributed transaction.. An…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,298 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2023-01-12T12:10:16.01+00:00
Pragna Venkata Bhanu Damerla 0 Reputation points
answered 2023-02-02T14:38:21.1966667+00:00
ShaikMaheer-MSFT 28,106 Reputation points Microsoft Employee
0 answers

Is it possible to connect an Apache Spark Pool in an Azure Synapse workspace to an artifacts feed in Azure Devops?

I'm trying to configure an environment that my team are going to use with the Apache Spark pool that we've defined in our Synapse workspace. We have a number of internal packages that we have built and deployed to our own artefacts feed in Azure Devops…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,621 questions
asked 2023-02-02T12:21:09.1533333+00:00
Nick Petzold 0 Reputation points