Azure DevOps pipeline suddenly start failing!
2024-07-09T23:11:19.7718791Z Complete getting Artifacts From Template 2024-07-09T23:11:19.7719155Z Start deploying artifacts from the template. 2024-07-09T23:11:19.7722609Z Deploy dev_keyVault_linkedService of type…
synapse notebook
Hi , how we can add link services created in synapse in notebook available azure synapse link servicename :test2 select * from test1.table1
Migration to Azure
Hi friends, we have a requirement to Migrate Denodo to Azure, and I have these questions: What could be the challenges we can anticipate and be aware of? Does the data's size impact the migration's complexity and duration? What tools are the best suited …
Can I create external views in synapse from multiple data sources?
Hi, I have data from different azure regions and my idea to decrease overhead is to use just synapse workspace with multiple external data sources and create the external tables for each of them, since I can't put all the data into a same location for…
![](https://techprofile.blob.core.windows.net/images/u-dk0HQtVEKGDLrjF9P49A.png?8DAFCD)
![](https://techprofile.blob.core.windows.net/images/u-dk0HQtVEKGDLrjF9P49A.png?8DAFCD)
Static Code Analysis for ADF or Synapse Pipelines in Azure
Hi team, I would like to know is there any use case possible to perform static code analysis of our ADF or Synapse pipelines residing in Azure DevOps repository? Thanks, Debajit Mitra
synapse dataframe writing to oracle with JDBC never quits though it has finished writing. It results into a pipeline failure. sometimes similar code works fine.
Py4JJavaError: An error occurred while calling o4089.save. : java.sql.SQLRecoverableException: IO Error: Connection reset by peer at oracle.jdbc.driver.T4CConnection.logoff(T4CConnection.java:1050) at…
unable to delete the Apache Spark pool in Azure Synapse Analytics
unable to delete the Apache Spark pool and attached error messages. I tried to delete via powershell as well from Azure Cloud Shell but no luck. As you can see in the screenshot there is no notebook1.
merge statement in 2 data frame
Hi , how to use merge statement in 2 dataframe df1=spark.sql("" sellect cole1,col2 from table1""") df2=spark.sql("" sellect cole1,col2 from table2""") expected results merge into table2 using tabl1 on…
Issues Connecting to Infor Data Lake Using Spark JDBC in Azure Synapse Spark Notebook
Hi Team, I'm encountering difficulties while attempting to connect to Infor Data Lake using Spark JDBC within Azure Synapse. Despite configuring the necessary JDBC driver and ionapi file path, I consistently encounter the following…
multiple lookup in copy activity
Hi, how to handle multiple lookup column as input in copy activity
Extracting Purchaser Data from Azure Synapse for Marketing Automation
I'm looking to connect Azure Synapse Analytics with a marketing automation platform. My goal is to identify recent purchasers from my existing Azure Synapse data and send them targeted emails using the marketing platform. While I can handle sending…
AZ DATA FACTORY - COPY Activity : 'Incorrect syntax near '>'
Hi, while i am trying to copy data using below dynamic value. SELECT * FROM @{pipeline().parameters.P_DS_Object_api} WHERE lastmodifieddate >= '@{formatDateTime(activity('watermarktable').output.firstRow.Watermarkvalue, 'yyyy-MM-dd HH:mm:ss')}' i am…
Notification email fail/not started pipeline Synapse
I would like to set up a notification (email) for a pipeline in Synapse to be sent if it fails or if it does not start. How do I do this? I have already created an Azure alert, but it only triggers if the pipeline encounters an error, not if it does not…
Guidance on how to use Service Principal with Certificate to Authorize for EventHub Stream Read
I found this documentation https://github.com/Azure/azure-event-hubs-spark/blob/master/docs/use-aad-authentication-to-connect-eventhubs.md online on how to use service principal with certificate to use spark stream read from EventHubs, I want to do this…
Azure Synapse LinkConnection template json
We have created a LinkConnection in Azure synapse, the source database is MS SQL Server and target database is Dedicated SQL Pool. We are trying to create powershell script to automate the creation of this LinkConnection for each environment - Testing,…
How to install com.crealytics.spark.excel to Synapse Spark Pool
Hello, I have spark pool with following properties: According to the instruction mentioned under the Q&A https://learn.microsoft.com/en-us/answers/questions/1185314/how-to-write-data-from-notebook-to-dedicated-sql-p?cid=kerryherger I have…
Azure Synapse CETAS is not working
Hi, When creating an external table in Synapse SQL Pools (SSP) using the following query, I encountered an error: "Operation failed since the external data source 'https://<storageacc>.dfs.core.windows.net/<containername>/asa' has…
Azure Synaps serverless scan file that should be filtered
Hey, We have a large amount of files stored in Storage Account in Parquet format with the following hierarchy. /m_id=%M_ID%/quarter=%QUARTER%/month=%MONTH%/partition_version=last/part-0.parquet'. examples for quarter values: 2024Q1, 2024Q2, 2024Q3…
Import filenames from Azure Storage container to SQL Table using Azure Data Factory
Hi All, I am trying to import filenames from an Azure Storage container into a table in a Dedicated SQL Pool (formerly SQL DW) using Azure Data Factory. For example, I have around 2,000 JSON files in an Azure Storage container, and I would like to…
unable connect Azure Synapse database
Hi, while creating link services for connecting azure synapse database..i am getting error that Insufficient permissions to call this API. 1829be47-97cd-4a80-b8f7-9402f8fbcc20 does not have Microsoft.Synapse/workspaces/credentials/useSecret/action on…
![](https://techprofile.blob.core.windows.net/images/VfQFAmOikEWfBHko2XlWTA.png?8D7F33)