how to query trigger runs on Azure Synapse Analytics using rest api?
https://learn.microsoft.com/en-us/rest/api/datafactory/trigger-runs/query-by-factory?view=rest-datafactory-2018-06-01&tabs=HTTP I tried a similar process in Azure synapse analytics like above one. But I got an error like below. it shows object id…
Unable to deploy code into another synapse workspace where Linked Service and Integration runtime are different
Hello I have azure synapse workspace and ETL code on that workspace. I need to deploy the same code on new workspace and new subscription/tenant. I am using Synapse deployment (template and params file). I created yaml file where I can run the…
Do we need to take any action if we use "AutoResolveIntegrationRuntime" in azure synapse analytics
Azure Synapse Analytics Compute Optimized data flows will be retired on 31 August 2024 – transition to General Purpose data flows
Dynamic Content Option for Password Field in SFTP Connector in Azure Synapse Analytics
I am working on a feature in Azure Synapse Analytics that connects to different SFTP servers dynamically based on a meta-data table. As I proceeded through it, I found out that in SFTP Linked Service, the PASSWORD field does not allow dynamic content.…
Changing the executor count in the Spark job definition configuration in Azure Synapse Analytics is not being reflected in the job
Hello All, Since my job was running for a long time, I did some analysis and decided to change the executor count to reduce the job running duration. My previous executor count was 4, and I tried to scale it up to 6. But the issue is, I could only…
Azure Synapse fails to load Lake Databases (500 error)
When loading a Synapse workspace, it sits at "loading resources" for awhile and eventually errors with a 500 for lake databases. The error is not particularly helpful. Any ideas on how I could potentially determine if it's a single data lake…
How to flatten an Array json using Azure Data Flow
I am tying to convert the following json to parquet file in azure synapse analytics, to get data from rows[]. The json response has : dimensionsheaders , metricheaders and rows, Where the dimensionsheaders and metricheaders hold the field names and…
Is it possible to use Synapse link to retrieve attachments from Dataverse?
Hello, I noticed that currently Synapse link for dataverse does not sync attachments from dataverse. The only way to get attachments is to make Dataverse REST API calls. Is there a way to use synapse link to get the attachments as well ?
Synchronizing a Synapse table from an on-premise Oracle table
Hi, I've a Synapse table on a dedicated SQL pool and I need to update it reading from an Oracle table. I've already implemented a Self-hosted IR and I've already read from the Oracle table rightly with the Oracle connector. The requirement from my…
ORA-12650: No common encryption or data integrity algorithm - connectivity error in Synapse.
Hi Team, We are using an Azure Self Hosted Integration Runtime to connect to an on-premise Oracle SQL Database, we had pipeline running everyday till yesterday before the west europe outage ( Ticket ID - RN9S-39Z ). Post the outage fix we have been…
Enable Managed Private EndPoint for existing synapse resource
Hi all - We're seeking to activate MPN for an existing Synapse resource, but we're unable to locate the option via the UI. Please refer to the attached screenshot. Could you kindly advise if there are any available options to enable MPN for the existing…
I can't do multiple unroll in flatten activity synapse dataflow
It was working quite well before, And started failing with "none.get" which is no where helping me to find the root cause, Even though I figured out that multiple flatten Unroll by cause the issue. My source is multiple Nested JSON try to…
Mssparkutils.credentials.getSecretWithLS to get Certificate and create CertificateCredential in Synapse
I need to create a CertificateCredential from azure.identity, I have a PKCS12 certificate saved in a Key Vault and I am reading it using mssparkutils with LinkedService, official documentation mentioned you can create a CertificateCredential with the…
Apache Spark in Azure Synapse Analytics - HTTP request in notebook
Hello. I use a Notebook in Synapse where I run my Python code. I would like to make an API request from this Notebook to Microsoft Purview to send the entities. I added the pyapacheatlas library to spark. On my local computer, this code works fine…
Azure Synapse studio 403 issue when public network disabled
We have created azure synapse studio with public access disabled and we have created private end points for synapse workspace with private link hub to access web synapse. but still we are getting 403 forbidden error while opening synapse web UI.
Federated Identity credentials auth in azure table sdk
Hi I was trying to use FIC creds for table sdk but was getting the following issue: CredentialUnavailableError: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint. This is the doc I am following for FIC…
Granting access to collections of objects within a workspace.
Goal: Achieve visibility and access control over pipelines, scripts, and data to the respective departments only in Azure Synapse Analytics workspace. Additional details: Data: Stored in Azure Synapse dedicated SQL pool. Description: I have data tables,…
Notebook fails in pipeline due to error 6002 , user error
**Notebooks function well when run normally, however they malfunction when utilized in a pipeline. The cause of the error is data write in csv, however the reason for the failure is unknown. ** Operation on target Notebook 3 failed:…
Error publishing notebook.
I am trying to publish my notebook. I get the following error : CreateOrUpdateNotebook failed: BadRequest. The operation failed because the entity “notebookA” is being renamed to “notebookB”. A retry of rename from exact source to target may work”. It…
Unable to deploy code to different Synapse workspace when LinkedService and ItegrationRun time are different
Hello I need help with smth and this is the current setup: Have many synapse Pipelines on subscription A I have created another subscription and workspace on another client tenant/subscription B. I created linked service and Integration Runtime on…