Content
Cannot get secret from Azure synapse integrate pipeline
I'm trying to create a Synapse pipeline that executes a notebook which depends on a secret that I'm getting from mssparkutils.credentials.getSecret. When I run the notebook manually from Develop I don't get any errors, but when I try to execute the…


Can we convert R script to M code (Power Query)
As we are having few scripts in R which we are consuming in another scope, as per the new scope that needs be migrated to M (Power Query). If so, are there any best practices to migrate existing scripts to M code. Do we have any tools that can be used…


How is the divison in terms of Spark applications submitted from a pipeline handled?
In a Synapse pipeline I have 4 different notebooks that are attached to the same Spark pool with 1 executor each. The spark pool has 3-8 nodes so it should have enough capacity to run all notebooks on the same Spark Instance. The pipeline has a trigger…


Azure Synapse Workspace - Role and Permission to see SQL views and execute views
Hi, I have data lake storage account which has raw JSON files. I created Synapse Workspace and added SQL serverless database which has 2 different views on top of these JSON files. I as admin can query this views and able to see results in synapse…


Using getSecret() to retrieve a secret value from KeyVault in Gov cloud
For public cloud we are using getSecret(akvName: String, secret: String, linkedService: String) to get a secret from keyvault. However, as we are setting up our government cloud, the host this uses is the public cloud<keyvault>.vault.azure.net…
ForEach activity to loop through an SQL parameters table?
I'm working on an ETL pipeline in Azure Synapse. In the previous version I used an Array set as a parameter of the pipeline and it contained JSON objects. For example: [{"source":{"table":"Address"},"destination…


Failed to fetch billing info
As part of a new project, I am setting up a resource group with all relevant services. One such service is Azure Synapse Analytics which is deployed and ready to go. For this particular project we are going with notebooks, hence we need an Apache Spark…


How to include a new incremented column using DENSE_RANK() in Synapse
In my synapse, I have a schedule table that stores information about all programs for a given day. It would be helpful if I could include rank/dense_rank in the output from this table based on the title and ordered by the event number and program date.…


Does Interactive authoring of the integration runtime have to be activated manually?
Hello! I just wanted to know if there's a way to automate the enabling of the interactive authoring of the AutoResolveIntegrationRuntime. Thanks!


Cannot find the CREDENTIAL '<db_scoped_creds>', because it does not exist or you do not have permission
I created external tables using the steps specified here: https://learn.microsoft.com/en-us/azure/synapse-analytics/sql/develop-storage-files-storage-access-control?tabs=shared-access-signature. Within Synapse workspace, I am able to access external…


Getting Error with Polybase in Copy Command and at same time Bulk Insert works?
Hi, I was getting below mentioned error with polybase, and unable to load data but the same file is getting loaded with bulk insert PolybaseOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Error happened when…


ADF Foreach Activity- Suggest on settings to exclude successful iterations in rerun
Now, you can see where your pipeline is failed it allows you rerun from that activity, which includes next iteration in a for loop which is successful. Behavior for re-run is valid in case of synchronization option, but in parallel scenario it's not…


How to Secure Input in Webhook Activity Synapse/ADF
Likewise in Web Activity, how to secure input in Webhook Activity either from synapse/ADF? I would like to secure this highlighted URL since it is a webhook token URL. Scenario: We are triggering runbooks from ADF/Synapse pipelines by using…


Dynamic load balance
We have a Synapse Data Factory process that dynamically kicks off >200 extracts. As Data Factory has a 20 pipeline running concurrently limit, the remaining extracts are queued behind the initial 20 However, it appears to be evenly queuing the…


Is it possible to have multiple Serverless sql pool in azure Synapse?
Hi Team, I am able to create Multiple Dedicated SQL Pool in Azure . But i am unable to create new serverless sql pool in azure synapse. There is no option to choose the pool type in sql pool creation. Please advice. Thanks!


Unable to access Azure Synapse Resource Overview - suggested actions?
Is anyone familiar with the following issue? Upon attempting to access the Azure Synapse resource, I receive the following error when navigating to the resource overview page: "The extension encountered an unexpected error and provided no additional…


Managed Resource Group Billing
Hello, I am trying to set billing reader role to Synapse Managed Resource Group but it has a Deny that does not let me to set this grant. If I look for billing information for this resource group at subscription scope I can not find anything, but I can…


Synapse Workspace in West Europe not opening up
We have 3 synapse analytics workspaces in West Europe Region, and have been not able to open them up since today morning.


in "Configure Conditional Access" in module "Secure a data warehouse in Azure Synapse Analytics" appears problem in page MS learning
hi there i met a problem to open this website in MS learn i followed the Instructions he is gives me but still not work and i try from other website like google and MS edge but still same


Logging of Debug Pipeline Runs
Hi I'm setting up an alert rule in Log Analytics to notify when a Synapse pipeline has been running for more than 2 hours. We would like to be notified of both pipeline runs executed by trigger and debug runs. However, in my testing, it seems as if…

