Microsoft Q&A

Azure Synapse Analytics

2,884 questions

An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.

Browse all Azure tags

2,884 questions with Azure Synapse Analytics tags

Sort by: Updated
Updated Created Answers
0 answers

How to use linked service in Notebook with pyspark

I have pyspark script in Notebook to read and write data in ADLS Gen2. Below is an sample of the pyspark script. But in the Synapse I only have a linked service created with Service Principle could connect to the ADLS Gen2, so I need to specify in…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
939 questions
asked 2023-03-31T04:24:40.7333333+00:00
Shao Peng Sun 31 Reputation points
edited the question 2023-03-31T16:43:08.6566667+00:00
deherman-MSFT 25,626 Reputation points Microsoft Employee
0 answers

Azure Synapse Analytics - Serverless SQL Pool - Queries Considerably Slow Down When Running Concurrently

I'm mostly new to the world of "big data", but have a good enough understanding of how data lakes, SQL pools, etc work and how they come together. However, I've setup an Azure Data Lake Gen2 storage account, being queried by an Azure Serverless…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
asked 2023-03-29T14:37:07.8466667+00:00
elbar78356853 0 Reputation points
commented 2023-03-31T16:32:52.3633333+00:00
AnnuKumari-MSFT 21,406 Reputation points Microsoft Employee
0 answers

Error with SSIS OLE DB Connection to Azure synapse serverless pool

Hello Microsoft Support Community, I am looking to pull data from a view in a serverless SQL pool in an azure synapse workspace. To connect to and preview the data, I am using an on-prem SSIS package with an OLE DB connection manager. I am able to…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
247 questions
asked 2023-03-31T16:30:41.8133333+00:00
Travis Wolf 0 Reputation points
1 answer

How do I configure Azure Cognitive Search datasource to connect using service principal?

I am moving from SQL server to Synapse and need to connect my data source in Cognitive Search using a service principal. I do not see an option for this. Is it possible?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
Azure Cognitive Search
Azure Cognitive Search
An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.
218 questions
asked 2023-03-30T21:43:17.9+00:00
Hefner, Craig 0 Reputation points
commented 2023-03-31T16:17:19.6766667+00:00
Grmacjon-MSFT 9,581 Reputation points
1 answer

Can you give us some more details on how to do this?

Can you give us some more details on how to do this? https://learn.microsoft.com/ja-jp/azure/data-factory/data-flow-lookup#analyzing-matched-rows

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
asked 2023-03-30T08:29:37.44+00:00
NishimuraChinatsu-9854 541 Reputation points
answered 2023-03-31T16:12:11.5933333+00:00
AnnuKumari-MSFT 21,406 Reputation points Microsoft Employee
1 answer

How do you resolve errors in variable contents?

Please let me know how to solve this. I know what the cause is, but I can't think of a way to fix it. The activity structure of synapsen is the image and JSON shown above. The pipeline and data structure and JSON code are as shown in the URL below. …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
asked 2023-03-29T07:22:35.4266667+00:00
NishimuraChinatsu-9854 541 Reputation points
commented 2023-03-31T15:53:35.6933333+00:00
AnnuKumari-MSFT 21,406 Reputation points Microsoft Employee
2 answers

How to set a maximum limit of spending on azure

Hello, I need your help to understand how I can set a maximum budget limit on my azure account. For example, let's say I don't want to spend more than 200 dollar per month how can I set that?  thank you

Azure SQL Database
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
Azure App Services
Azure App Services
A feature of Azure App Service used to create and deploy scalable, mission-critical web apps.
4,607 questions
Azure Logic Apps
Azure Logic Apps
An Azure service that automates the access and use of data across clouds without writing code.
1,980 questions
Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
2,811 questions
asked 2023-03-30T14:18:28.18+00:00
alex smail 0 Reputation points
commented 2023-03-31T15:38:44.45+00:00
Grmacjon-MSFT 9,581 Reputation points
0 answers

Synapse - UserAccounts

On Azure Synapse, We want to control using the workload management by using resource limitations. Can we use something like OAUTH token for multiple teams instead of having multiple accounts to trace the usage and queries?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
asked 2023-03-22T14:45:42.18+00:00
mrsolo 0 Reputation points
commented 2023-03-31T14:37:11.59+00:00
mrsolo 0 Reputation points
1 answer

is it possible to query a data lake with a dedicated sql pool in Azure like you can with a serverless sql pool?

is it possible to query a data lake with a dedicated sql pool in Azure like you can with a serverless sql pool?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
Azure SQL Database
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,721 questions
asked 2023-03-30T19:52:08.8933333+00:00
Tran, Peter 21 Reputation points
edited an answer 2023-03-31T14:12:41.7466667+00:00
GeethaThatipatri-MSFT 17,211 Reputation points Microsoft Employee
0 answers

getting error - Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Setting QUOTED_IDENTIFIER to 'OFF' is not supported..

when trying to execute procedure in synapse dedicated pool from local command prompt as attached and getting error as - Sqlcmd: Error: Microsoft ODBC Driver 17 for SQL Server : Setting QUOTED_IDENTIFIER to 'OFF' is not supported.. Is this related to any…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
asked 2023-03-31T13:46:37.43+00:00
Dhanya Gopan 0 Reputation points
1 answer

How to best mimic system versioned tables in Synapse Dedicated Pool when migrating from Azure SQL Server?

We are looking to migrate a database from Azure SQL Server to Synapse Dedicated pool. We use system-versioned tables (aka Temporal tables) in Azure SQL Server. Synapse Dedicated Pools do not support system versioned tables out of the box it seems. Any…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
asked 2023-03-24T08:30:53.97+00:00
Derek Horrall 141 Reputation points
commented 2023-03-31T12:12:55.5833333+00:00
Derek Horrall 141 Reputation points
1 answer

Copy Dataverse data from ADLS Gen2 to Azure SQL DB leveraging Azure Synapse Link template in Synapse not working for multiple entities

Hi, While using a Template related to Copy data from Dataverse to SQL via Azure Synapse link, templates while configuring for multiple entities are not working. Issues I was facing After configuring Dataflows, pipelines are working successfully, but…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
asked 2023-03-27T12:04:42.96+00:00
CHARAN TEJA NALLURI 1 Reputation point
edited the question 2023-03-31T11:53:48.0033333+00:00
CHARAN TEJA NALLURI 1 Reputation point
2 answers

How to increase size of varchar which is already set to 8000 and if we try to set as varchar(max) it is showing error likeThe statement failed. Column 'column name' has a data type that cannot participate in a columnstore index.

using these to run pipelines

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
SQL Server
SQL Server
A family of Microsoft relational database management and analysis systems for e-commerce, line-of-business, and data warehousing solutions.
9,172 questions
asked 2023-03-24T12:39:10.2266667+00:00
Priya 0 Reputation points
commented 2023-03-31T09:28:42.0166667+00:00
CosmogHong-MSFT 9,161 Reputation points Microsoft Vendor
1 answer

rowcount from parquet file in synapse pipeline

HI - I need to know the row count from parquet file in synapse pipeline. I am trying with lookup activity but it has some limitation in keeping the rows in memory and throws an error if the data size exceeds 1MB(?). Is there any better way to find it…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
asked 2023-03-30T16:47:50.54+00:00
rajendar erabathini 586 Reputation points
answered 2023-03-31T09:20:22.8866667+00:00
ShaikMaheer-MSFT 30,426 Reputation points Microsoft Employee
1 answer

How do I view the Dedicated pool as a Azure SQL DB

Hi, I have a Stream analytics job. I need to bring one reference input from a Synapse dedicated pool. Can I view or configure the dedicated pool as a Azure SQL DB so that I can bring the data in my ASA job (as the ASA only allows Azure SQL DB). Thanks…

Azure SQL Database
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
237 questions
asked 2023-03-16T12:45:27.1066667+00:00
Anonymous
edited a comment 2023-03-31T07:55:39.7133333+00:00
Subashri Vasudevan 5,101 Reputation points
0 answers

Nested Execute Pipeline issue in ADF master pipeline

Hi ,In my case i have a master pipeline which has nested Execute pipelines like execute1 pipeline contains lookup and foreach inside foreach execute2 .inside execute 2 get metadata and and foreach again inside foreach execute3 .execute3 contains lookup…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,721 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
939 questions
asked 2023-03-30T14:36:54.07+00:00
Prapul Kumar Dongari 71 Reputation points
commented 2023-03-31T07:13:28.0033333+00:00
ShaikMaheer-MSFT 30,426 Reputation points Microsoft Employee
0 answers

ServiceBus_CDMToSynapseView' was unable to start

I am deploying the components to read the csv and metadata files landing the azure datalake from the Dynamics 365 finance and Opertions system that create tables in the synapse server. I can see the folder and files in the blob storage container. I…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
Azure Service Bus
Azure Service Bus
An Azure service that provides cloud messaging as a service and hybrid integration.
365 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
1,695 questions
asked 2023-03-31T06:40:41.9666667+00:00
Wayne Bartkowski 586 Reputation points
0 answers

azure synapse incremental pipeline duplicates at sink

Hello, I have built a simple incremental pipeline that helps me extract each file name and takes the latestfile and does a simple upsert with expression true(). I noticed the sync table comes in with duplicate values and I made sure my sink skips…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
asked 2023-03-31T06:39:22.8766667+00:00
Yvette Epingo 0 Reputation points
0 answers

Dynamics 365 FO CDMUtilSolution no tables appear in synapse

I am deploying the components to read the csv and metadata files landing the azure datalake from the Dynamics 365 finance and Opertions system. I can see the folder and files in the blob storage container. I deployed the CDM App in Azure, using the…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
asked 2023-02-28T03:21:06.72+00:00
Wayne Bartkowski 586 Reputation points
commented 2023-03-31T06:29:34.1666667+00:00
Wayne Bartkowski 586 Reputation points
1 answer

Copy Activity goes into Internal Server 500 error for rest identity.

Hello, I am using copy activity and selecting rest_identity for call data from different API. I made a main and micro pipeline. The main pipeline consists of calling the token which is being passed to several micro pipeline. Out of 5 micro-pipeline,…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,884 questions
asked 2023-02-20T13:40:48.99+00:00
Jadhav, Karan 10 Reputation points
commented 2023-03-31T05:30:31.78+00:00
PRADEEPCHEEKATLA-MSFT 56,551 Reputation points Microsoft Employee