Error in Pipeline: Failure happened on 'destination' side. ErrorCode=DeltaNotSupportedAction
My colleague and I keep running into this error in our pipeline. From our understanding we don't do anything with Meta Data. Does anyone have a resolution? Failure happened on 'destination' side.…
Forbidden 403 when trying to create a delta table in oneLake with notebook using sql
Created a spark dataframe from the csv files and also created a view TableTempView on top of this dataframe. When I try to create a table in the lakehouse using below code %%sql create table if not EXISTS Bronze_Sales21 using DELTA as select * from…
SQL Query to calculate size of each schemas in Azure data warehouse and to find last access time?
Hi, Is there any SQL query that could help me get the size of each schemas present in our Azure Data Warehouse service and also to find when was it last accessed? I checked and found one query below which can help me find the size of each table in…
Which role allows to add a client IP as a firewall rule for a Synapse Analytics workspace
Hi, I need to know which is the Synapse RBAC role or Azure role allows to add a client IP as a firewall rule for a Synapse workspace. Thanks
CI/CD Azure Synapse Analytics with GitHub Actions
Trying to create a CI/CD process using GitHub Actions without having to click the publish button in a synapse workspace, as well as, merge between different environments. Is it possible to do via GitHub actions without having to use Azure DevOps?
Flatten a JSON Nested Array in Azure Synapse pipeline
I am trying to flatten a JSON which has nested array objects with 1:M mapping. Please refer below JSON response. { "result": [ { "data": [ { "dimensions": [ …
How do I get a notification for Azure Synapse Workspace Publishes
We have an Azure Synapse Workspace in live mode. We do have a DevOps release pipeline that is used to deploy changes to the pipelines/artifacts. The aforementioned Workspace that is used as the target workspace for release deployments is kept in live…
![](https://techprofile.blob.core.windows.net/images/hAFzqf2_AwAAAAAAAAAAAA.png?8DB62B)
How getting the client IP to add as a firewall rule for Synapse Analytics
Hi, I need to get the client IP to add a firewall rule for Synapse. Which is the command to use, please? Thanks
Enabling auto update statistics for a Synapse dedicated SQL pool
Hi, I'd like if and when it is proper to enable the auto update statistics option for a Synapse dedicated SQL pool. Thanks
Synapse Integration Runtime with public subtype when Managed virtual network is enabled
Hello, I have created a synapse analytics workspace with managed virtual network enabled. This is so that I can use private endpoints to access a sql database that has public access disabled. The access to this resource uses a newly created integration…
Can you give me example for shared responsibility and service model related to like facebook?
I started learning azure i cannot see the big picture how it works.can i get an example like how facebook or instagram works in cloud computing based on their shared responsibility and their service model?
How to find who updated the Azure Synapse sql database objects
Hi All, We have granted access to the Active Directory User groups in our Azure Synapse SQL database objects. I can see the AD group which has created or updated a database object, I am trying to see which individual user has done the change. Can someone…
Synapse troubleshoot %run in live vs debug
Hello, I am having difficulty getting %run notebook to work in Synapse live for notebook not sharing the same path. I do not have problems when the path is the same. I have tried no quotes and with quotes. I have tried path with a "/" in front…
![](https://techprofile.blob.core.windows.net/images/sm85N7XqFEOpiNbDnxvFcg.png?8DA01F)
Optimizing Azure Synapse Analytics for Spark Pool and Mapping Data Flows
I am encountering errors involving Livy sessions when running multiple pipelines in Synapse Notebook and mapping data flows. The errors are as follows: For notebooks: Operation on target forecast_constrained failed: Exception: Failed to create Livy…
What is the reason for Azure synapse pipeline activity is in queue for a longer time?
Just like few days back the issue Data Factory failures or pipeline run delays in East US is back in East US. The Pipelines are all queued up and failed to enable the interactive authoring capability.
![](https://techprofile.blob.core.windows.net/images/hAFzqf2_AwAAAAAAAAAAAA.png?8DB62B)
How to create and use separate Spark sessions/apps within the same notebook session?
We process data in various logical steps from raw to curated where we persist intermediate results for each of the steps and the following steps starts with reading the predecessor results (similar to medallion architecture pattern). This allows us to…
Azure Synapse Error: This query cannot be executed because we can't get enough capacity for optimal performance in time. This can be an intermittent issue. Please retry later.
I am receiving this error message since yesterday afternoon when running a Synapse Pipeline that is using SQL Serverless. The Pipeline usually runs twice a day without any issues. [This query cannot be executed because we can't get enough capacity for…
Binding errors in azure synapse studio when i try querying the view
We have recently facing issues when trying to run the query on top of a parquet file for view or an external table. Nothing has changed in our setup. This issue seems to come and go after a few minutes; we are able to query the view as normal, but this…
![](https://techprofile.blob.core.windows.net/images/y9NbTUj9TEaQZy_QzgrIrg.png?8D99D5)
Synapse Analytics workspace not deploying to separate workspace using GitHub Actions
My synapse analytics workspace is integrated with GitHub, dedicated sql pools are the exact same between development workspace and uat workspace. Target workspace is indicated within the yaml files yet none of the published ARM templates will publish to…
How does Azure Data Factory's Data Copy REST API pagination rules work, is there any parallelisation and can this be disabled to only copy one page at a time?
Hello, I'm using the copy data activity in a Azure Data Factory pipeline to copy data from a REST API data source (the pipeline will soon be migrated to a Synapse one). A recent update to the source API has stumped me as using the built in pagination…