Microsoft Q&A

Azure Synapse Analytics

2,600 questions

An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.

Browse all Azure tags

2,600 questions with Azure Synapse Analytics tags

Sort by: Updated
2 answers

Cannot get secret from Azure synapse integrate pipeline

I'm trying to create a Synapse pipeline that executes a notebook which depends on a secret that I'm getting from mssparkutils.credentials.getSecret. When I run the notebook manually from Develop I don't get any errors, but when I try to execute the…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2022-06-07T09:44:49.417+00:00
Caio Cavalcanti 1 Reputation point
answered 2023-01-28T17:43:30.7733333+00:00
Miroslav Muras 0 Reputation points
0 answers

Can we convert R script to M code (Power Query)

As we are having few scripts in R which we are consuming in another scope, as per the new scope that needs be migrated to M (Power Query). If so, are there any best practices to migrate existing scripts to M code. Do we have any tools that can be used…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,261 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
Azure Analysis Services
Azure Analysis Services
An Azure service that provides an enterprise-grade analytics engine.
337 questions
asked 2023-01-27T12:38:18.8+00:00
Manohar Kondragunta 0 Reputation points
edited the question 2023-01-28T04:47:39+00:00
AjayBathini-MSFT 0 Reputation points Microsoft Employee
1 answer

How is the divison in terms of Spark applications submitted from a pipeline handled?

In a Synapse pipeline I have 4 different notebooks that are attached to the same Spark pool with 1 executor each. The spark pool has 3-8 nodes so it should have enough capacity to run all notebooks on the same Spark Instance. The pipeline has a trigger…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-18T16:16:47.5866667+00:00
asciibscii 20 Reputation points
edited an answer 2023-01-27T23:24:42.99+00:00
BhargavaGunnam-MSFT 7,311 Reputation points Microsoft Employee
0 answers

Azure Synapse Workspace - Role and Permission to see SQL views and execute views

Hi, I have data lake storage account which has raw JSON files. I created Synapse Workspace and added SQL serverless database which has 2 different views on top of these JSON files. I as admin can query this views and able to see results in synapse…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-19T22:47:52.4933333+00:00
Jaimin Kansara 0 Reputation points
commented 2023-01-27T20:20:32.8666667+00:00
Jaimin Kansara 0 Reputation points
2 answers

Using getSecret() to retrieve a secret value from KeyVault in Gov cloud

For public cloud we are using getSecret(akvName: String, secret: String, linkedService: String) to get a secret from keyvault. However, as we are setting up our government cloud, the host this uses is the public cloud<keyvault>.vault.azure.net…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-25T00:47:34.4733333+00:00
natalie 6 Reputation points Microsoft Employee
accepted 2023-01-27T19:44:18.4466667+00:00
natalie 6 Reputation points Microsoft Employee
1 answer One of the answers was accepted by the question author.

ForEach activity to loop through an SQL parameters table?

I'm working on an ETL pipeline in Azure Synapse. In the previous version I used an Array set as a parameter of the pipeline and it contained JSON objects. For example: [{"source":{"table":"Address"},"destination…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-23T10:28:06.7+00:00
Vasas, Monika 20 Reputation points
accepted 2023-01-27T18:39:21.5333333+00:00
Vasas, Monika 20 Reputation points
1 answer

Failed to fetch billing info

As part of a new project, I am setting up a resource group with all relevant services. One such service is Azure Synapse Analytics which is deployed and ready to go. For this particular project we are going with notebooks, hence we need an Apache Spark…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-26T09:21:54.4233333+00:00
Jens Vestergaard 0 Reputation points
edited a comment 2023-01-27T18:23:36.83+00:00
KranthiPakala-MSFT 32,896 Reputation points Microsoft Employee
0 answers

How to include a new incremented column using DENSE_RANK() in Synapse

In my synapse, I have a schedule table that stores information about all programs for a given day. It would be helpful if I could include rank/dense_rank in the output from this table based on the title and ordered by the event number and program date.…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-27T16:37:36.06+00:00
Vivek Komarla Bhaskar 406 Reputation points
edited the question 2023-01-27T16:39:00.82+00:00
Vivek Komarla Bhaskar 406 Reputation points
1 answer

Does Interactive authoring of the integration runtime have to be activated manually?

Hello! I just wanted to know if there's a way to automate the enabling of the interactive authoring of the AutoResolveIntegrationRuntime. Thanks!

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
SQL Server Integration Services
SQL Server Integration Services
A Microsoft platform for building enterprise-level data integration and data transformations solutions.
1,840 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,261 questions
asked 2022-06-29T16:07:00.26+00:00
commented 2023-01-27T15:53:05.7033333+00:00
Marc Scirri 1 Reputation point
1 answer One of the answers was accepted by the question author.

Cannot find the CREDENTIAL '<db_scoped_creds>', because it does not exist or you do not have permission

I created external tables using the steps specified here: https://learn.microsoft.com/en-us/azure/synapse-analytics/sql/develop-storage-files-storage-access-control?tabs=shared-access-signature. Within Synapse workspace, I am able to access external…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2021-07-28T04:07:55.333+00:00
Waqas Hussain 116 Reputation points
commented 2023-01-27T14:53:46.1966667+00:00
Agazoth 96 Reputation points
1 answer

Getting Error with Polybase in Copy Command and at same time Bulk Insert works?

Hi, I was getting below mentioned error with polybase, and unable to load data but the same file is getting loaded with bulk insert PolybaseOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Error happened when…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-24T03:55:15.2766667+00:00
Mutthuluru Yashwanth Sai 0 Reputation points
commented 2023-01-27T14:48:01.3866667+00:00
BhargavaGunnam-MSFT 7,311 Reputation points Microsoft Employee
1 answer

ADF Foreach Activity- Suggest on settings to exclude successful iterations in rerun

Now, you can see where your pipeline is failed it allows you rerun from that activity, which includes next iteration in a for loop which is successful. Behavior for re-run is valid in case of synchronization option, but in parallel scenario it's not…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,261 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-26T19:52:50.69+00:00
Goutham Kolasani 0 Reputation points Microsoft Employee
edited an answer 2023-01-27T13:51:09.8933333+00:00
AnnuKumari-MSFT 19,116 Reputation points Microsoft Employee
0 answers

How to Secure Input in Webhook Activity Synapse/ADF

Likewise in Web Activity, how to secure input in Webhook Activity either from synapse/ADF?     I would like to secure this highlighted URL since it is a webhook token URL.   Scenario: We are triggering runbooks from ADF/Synapse pipelines by using…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,261 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-27T07:42:10.1366667+00:00
Murali Relamadugu 0 Reputation points Microsoft Employee
edited a comment 2023-01-27T12:10:43.4033333+00:00
AnnuKumari-MSFT 19,116 Reputation points Microsoft Employee
0 answers

Dynamic load balance

We have a Synapse Data Factory process that dynamically kicks off >200 extracts. As Data Factory has a 20 pipeline running concurrently limit, the remaining extracts are queued behind the initial 20 However, it appears to be evenly queuing the…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,261 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-26T23:05:01.87+00:00
Ryan Abbey 1,061 Reputation points
commented 2023-01-27T10:04:20.1733333+00:00
Samy Abdul 2,991 Reputation points
1 answer One of the answers was accepted by the question author.

Is it possible to have multiple Serverless sql pool in azure Synapse?

Hi Team, I am able to create Multiple Dedicated SQL Pool in Azure . But i am unable to create new serverless sql pool in azure synapse. There is no option to choose the pool type in sql pool creation. Please advice. Thanks!

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-26T07:07:26.9166667+00:00
MSquare Systems 20 Reputation points
commented 2023-01-27T09:57:08.44+00:00
MSquare Systems 20 Reputation points
1 answer

Unable to access Azure Synapse Resource Overview - suggested actions?

Is anyone familiar with the following issue? Upon attempting to access the Azure Synapse resource, I receive the following error when navigating to the resource overview page: "The extension encountered an unexpected error and provided no additional…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
152 questions
asked 2023-01-23T10:20:35.97+00:00
Petrov, Dragomir 5 Reputation points
commented 2023-01-27T08:26:13.8533333+00:00
Petrov, Dragomir 5 Reputation points
1 answer

Managed Resource Group Billing

Hello, I am trying to set billing reader role to Synapse Managed Resource Group but it has a Deny that does not let me to set this grant. If I look for billing information for this resource group at subscription scope I can not find anything, but I can…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-23T09:36:49.9633333+00:00
Manuel Palacios Esteban 0 Reputation points Microsoft Employee
commented 2023-01-27T05:15:30.1133333+00:00
PRADEEPCHEEKATLA-MSFT 53,086 Reputation points Microsoft Employee
1 answer

Synapse Workspace in West Europe not opening up

We have 3 synapse analytics workspaces in West Europe Region, and have been not able to open them up since today morning.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-23T08:52:33.4833333+00:00
Jayachandran, Athul 21 Reputation points
commented 2023-01-27T05:14:16.4066667+00:00
PRADEEPCHEEKATLA-MSFT 53,086 Reputation points Microsoft Employee
1 answer

in "Configure Conditional Access" in module "Secure a data warehouse in Azure Synapse Analytics" appears problem in page MS learning

hi there i met a problem to open this website in MS learn i followed the Instructions he is gives me but still not work and i try from other website like google and MS edge but still same

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
asked 2023-01-19T14:07:19.6766667+00:00
commented 2023-01-27T05:04:25.58+00:00
PRADEEPCHEEKATLA-MSFT 53,086 Reputation points Microsoft Employee
1 answer

Logging of Debug Pipeline Runs

Hi I'm setting up an alert rule in Log Analytics to notify when a Synapse pipeline has been running for more than 2 hours. We would like to be notified of both pipeline runs executed by trigger and debug runs. However, in my testing, it seems as if…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,261 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,600 questions
Azure Monitor
Azure Monitor
An Azure service that is used to collect, analyze, and act on telemetry data from Azure and on-premises environments.
1,804 questions
asked 2023-01-17T15:05:16.8933333+00:00
Frederik Larsen 51 Reputation points
commented 2023-01-27T04:56:24.75+00:00
PRADEEPCHEEKATLA-MSFT 53,086 Reputation points Microsoft Employee