File cannot be opened because it does not exist or it is used by another process in Azure Synapse workspace
Hi, I created a new Azure Synapse Analytics resource and uploaded some csv files to the data lake storage created with the Synapse. I can see the files in the workspace and preview the files. However, when I tried to create a new SQL script from the…
Automatic updates on source systems
Hi friends, I have a requirement that the business has 12 different upstream sources SAP, CRM, Oracle HCM, SqlServer, PostgreSQL, and multiple other open-source databases and custom applications, etc. and every system has its master record, and just…
How to get notified when synapse integration runtime status is not in running state
Hello, I'm working on setting up a notification system in Azure Synapse Analytics to alert me when my self-hosted Integration Runtime (IR) is not in a 'running' state. My plan is to create a pipeline that retrieves the IR status through a REST API and…
Pass value from an array selectively to For Each
Hi, I have a parent pipeline that captures values in an array - ArrayFullLoadData. Currently ArrayFullLoadData have three values [A,B,C] and the same is passed to a ForEach with @pipeline.parameters.ArrayFullLoadData. Inside ForEach, values A, B,C are…
Azure Synapse Error
Hi everyone! I used Metada Driven toll from Azure Synapse Analytics in order to build my pipeline. It works perfectly fine, but onse in while I am getting this error msg. { "errorCode": "2200", "message":…
Synapse workspace deployment node-js supported versions
Hi, I would like to know the list of node.js versions supported for Synapse workspace deployment (Synapse Workspace Deployment@2 task). I can't seem to find any documentation describing the same.
Azure Synapse Analytics Workspace And its Dedicated SQL Pool Migration From one subscription to another subscription during Cross Subscription Migration
I just Want to know that how can I migrate synapse workspace and its dedicated SQL pool from one subscription to another subscription during cross subscription migration
Cannot register microsoft.Synapse
Hi, I am encountering a problem. No matter how many times I click 'register' on Microsoft Synapse, it always shows as 'NotRegistered'.
Azure Synapse alert for Dedicated SQL Pool
How can I properly set up an alert for a Dedicated SQL Pool in Azure Synapse for CPU Utilization? I want the alert to trigger if the CPU Utilization is more than 90% for an hour continuously. I have set the "check every" to 1 hour and…
How to upgrade the Azure Synapse Runtime for Apache Spark 3.1 to version 3.3
We received an email regarding the retirement of Apache Spark 3.1 as of January 26, 2024. There is a notice saying to upgrade to Apache Spark 3.3 when clicking on [Azure Synapse Link for Dataverse] from Power Apps, but I am not seeing the Update option…
InvalidHttpRequestToLivy: Your Spark job requested 12 vcores. However, the workspace has a 0 core limit.
it gives me this error and when i tried to increase core size limit i could not find that option so how to solve this error?
How to translate database content with Azure Translator by ADF or Synapse notebook?
There is an Azure Database table. Some of columns need to be translated from one language to another into additional columns. Such as from English to Spanish, or Portuguese to English, etc. I am exploring how I can use ADF or Synapse notebook to…
Backup/Restore and collation update in Azure Synapse
We have one requirement that to change the collation of Azure SQL serverless database in Azure Synapse Analytics to default collation. So I have 2 questions, 1. How to take backup and restore Azure SQL serverless database in synapse. 2. How to update…
No MediaTypeFormatter is available to read an object of type 'ClusterInfo' from content with media type 'text/html'.
Hi, in azure data factory I wanted to test the connection of our new Databricks Workspace URL and ClusterID. I parameterized the linked service (LS) to global parameters databricksURL and databricksClusterID. When I want to test the connection…
Cross-db query in Synapse
Hi, I am pretty new to Azure Synapse This post mentions that it is possible to have only one database per SQL pool. However, this post was from 2 years ago. Is this still the case? Assuming that only one database can be created within a SQL pool, is…
No includeDeletedObjects property on Salesforce as source in Copydata activity
Hi! I made the linked service according to the instructions: https://learn.microsoft.com/en-us/azure/data-factory/connector-salesforce?tabs=data-factory#linked-service-properties I set up the dataset according to the instructions. But when I use the…
Synapse Scala | Read Cosmos DB Container data using managed identity
Hi Team, I tried reading Cosmos data using Managed Identity inside a Synapse Scala notebook, but it's not working as expected. Already provided access for synapse workspace by using below command New-AzCosmosDBSqlRoleAssignment -ResourceGroupName…
rename table in synapse
Hi Team, rename table is not working in synapse RENAME OBJECT [dim].[test1] TO [dim].[test]; Parse error at line: 1, column: 49: Incorrect syntax near '.'.
Long queue time for copy activity in synapse pipeline
This Synapse Analytics pipeline runs a simple SQL script that joins a table with its change table and outputs the results (less than 100 rows) as a parquet file in Azure Data Lake Storage Gen2. A Copy activity with the SQL script as a source and Storage…
Can you assist with a Synapse issue involving memory allocation in SQL DW Copy Command in Data Factory, without solely resorting to increasing DWU?
I'm encountering an issue while using the Copy Activity in Data Factory with the Polybase copy option. The error message I receive is as follows: “…..SQL DW Copy Command operation failed with error 'Unable to allocate 257707 KB for columnstore…