Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,599 questions
1 answer
After some time, viewing pipeline log links lead to wrong page
This is something I consistently experience if I use the same Synapse Analytics Workspace for a decent amount of time: My workflow is: Monitor Pipeline Runs Click on a run of interest This shows me the pipeline that ran and all the individual steps…
asked 2024-06-27T14:13:06.36+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 95%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAC%3C/text%3E%3C/svg%3E)
Andrew Chung
20
Reputation points Microsoft Employee
answered 2024-07-01T01:29:13.68+00:00
Andrew Chung
20
Reputation points Microsoft Employee
1 answer
ALTER TABLE ADD COLUMN changes made to Synapse Delta tables in a Notebook do not reflect in Lake Database
Running the following spark.sql("ALTER TABLE example_table ADD COLUMN example_column STRING") Successfully adds the column to the Delta table and this can be verified by running %%sql SELECT example_column FROM example_table However when…
asked 2024-06-30T20:51:36.6366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 75%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFL%3C/text%3E%3C/svg%3E)
Francesca Lightbown
0
Reputation points
answered 2024-06-30T23:00:54.5066667+00:00
hossein jalilian
4,690
Reputation points
0 answers
Custom logging in Azure Synapse Datapipeline to to log analytics Workspace - Help Requested
Requirement:- Want to add custom logging in azure synapse pyspark job using python logging module and spark config Getting error while implementing the same in the pyspark job ModuleNotFoundError: No module named 'pyspark.SparkConf pyspark script for…
asked 2024-06-26T23:18:27.1633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 21%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJY%3C/text%3E%3C/svg%3E)
Johny Yagappan
20
Reputation points
commented 2024-06-30T18:06:46.4533333+00:00
Johny Yagappan
20
Reputation points
1 answer
Is Synapse Serverless Down?
As of ~12:53 CET all of our queries on Synapse Serverless have been timing out with the message: [This query cannot be executed because we can't get enough capacity for optimal performance in time. This can be an intermittent issue. Please retry…
asked 2024-06-27T12:05:26.4133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 17%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Mark Mellink
35
Reputation points
commented 2024-06-30T16:34:29.03+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
BhargavaGunnam-MSFT
28,446
Reputation points Microsoft Employee
0 answers
azure synapse
hey i try to get sdk call that return me the info about the Azure Active Directory i cant find the sdk call for this
asked 2024-06-26T08:13:52.66+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 56.00000000000001%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETM%3C/text%3E%3C/svg%3E)
Tommy Mordehay
0
Reputation points
commented 2024-06-30T15:11:24.2333333+00:00
PRADEEPCHEEKATLA-MSFT
83,886
Reputation points Microsoft Employee
0 answers
Unable to run an SQL query in Azure Synapse
Hi, I am unbale to run a simple sql query. It keeps throwing this following error. File 'https://storagecontainer/Financial Sample.xlsx' cannot be opened because it does not exist or it is used by another process. I assigned myself with Storage Blob Data…
asked 2024-06-30T14:25:48.4233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 17%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
shailaja Azure
0
Reputation points
edited the question 2024-06-30T15:10:28.12+00:00
Erland Sommarskog
104.8K
Reputation points MVP
1 answer
How to install com.crealytics.spark.excel to Synapse Spark Pool
Hello, I have spark pool with following properties: According to the instruction mentioned under the Q&A https://learn.microsoft.com/en-us/answers/questions/1185314/how-to-write-data-from-notebook-to-dedicated-sql-p?cid=kerryherger I have…
asked 2024-06-26T14:34:57.57+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 64%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETT%3C/text%3E%3C/svg%3E)
Tamashevich, Tatsiana
40
Reputation points
commented 2024-06-30T14:26:33.5733333+00:00
PRADEEPCHEEKATLA-MSFT
83,886
Reputation points Microsoft Employee
1 answer
How to send data into displayHTML function in Azure Synapse Analytics
Hi, As mentioned in this link https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-data-visualization I want to use D3.js to visualize my data. How do I send data from my pyspark dataframe into displayHTML() to visualize it?
asked 2024-06-28T06:28:54.47+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 45%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Raghul Kannan
126
Reputation points
edited an answer 2024-06-30T11:59:06.16+00:00
PRADEEPCHEEKATLA-MSFT
83,886
Reputation points Microsoft Employee
1 answer
Do Apache Spark Pools have a Service Tag?
Our company is rolling out more restrictive network security groups and I wanted to see if we could have our Apache Spark Pools' traffic be allowed via a service tag. When I tried to trace the IP address of the pools, the IPs varied widely so a static IP…
asked 2024-06-28T15:07:17.05+00:00
Andrew Chung
20
Reputation points Microsoft Employee
answered 2024-06-30T11:06:00+00:00
PRADEEPCHEEKATLA-MSFT
83,886
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Issue with running queries on Synapse SQL Endpoint - This query cannot be executed because we can't get enough capacity for optimal performance in time.
Dear Community, I am getting the error ""This query cannot be executed because we can't get enough capacity for optimal performance in time."" when running a query using a SQL Serverless endpoint in Synapse Analytics This error…
0 answers
Install Packages on Azure Synapse DEP-enabled workspaces
Hi I have a Synapse workspace with DEP enabled. Since PyPI libraries cannot be installed directly in the Spark pool, could you please advise on how to install the azure-mgmt-kusto, azure-kusto-data, and azure-keyvault-secrets packages on the pool? I…
asked 2024-06-26T02:17:08.77+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(160, 73%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAD%3C/text%3E%3C/svg%3E)
Abhiram Duvvuru
191
Reputation points Microsoft Employee
commented 2024-06-28T21:39:58.74+00:00
Abhiram Duvvuru
191
Reputation points Microsoft Employee
1 answer
Unable to run query on Synapse Serverless SQL Pool suddenly
I have a few of external table in an Azure Synapse Analytics Serverless SQL Pool which I need to query frequently. I suddenly got this error today. [This query cannot be executed because we can't get enough capacity for optimal performance in time. This…
asked 2024-06-27T06:30:27.38+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 12%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKW%3C/text%3E%3C/svg%3E)
Kenny Wong (HK)
40
Reputation points
answered 2024-06-28T21:12:33.17+00:00
BhargavaGunnam-MSFT
28,446
Reputation points Microsoft Employee
1 answer
Azure Synapse Error :This query cannot be executed because we can't get enough capacity for optimal performance in time. This can be an intermittent issue. Please retry later.
Strangely since morning Synapse SQL Serverless on one environment is not able to query external data in a storage account. This was running fine till now [This query cannot be executed because we can't get enough capacity for optimal performance in time.…
asked 2024-06-28T13:39:30.8133333+00:00
amit karmakar
0
Reputation points
answered 2024-06-28T21:10:21.5133333+00:00
BhargavaGunnam-MSFT
28,446
Reputation points Microsoft Employee
1 answer
Synapse how to read excel with formula
Hello, in Synapse spark pool we need to read excel file that contains formula inside of cells (sometimes it is simple sum, sometimes complicated formula, sometimes it is reference to another spread sheet and cell). Unfortunately, we observe that some…
asked 2024-06-27T08:57:56.29+00:00
Tamashevich, Tatsiana
40
Reputation points
commented 2024-06-28T12:19:38.78+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 4%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Sina Salam
6,341
Reputation points
1 answer
Import filenames from Azure Storage container to SQL Table using Azure Data Factory
Hi All, I am trying to import filenames from an Azure Storage container into a table in a Dedicated SQL Pool (formerly SQL DW) using Azure Data Factory. For example, I have around 2,000 JSON files in an Azure Storage container, and I would like to…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 74%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
0 answers
Azure synapse link creates model.json with no partitions
Hi, I am following this link below to copy dataverse data to Azure SQL DB using azure data factory pipelines. The model.json file created by my synapse link for dataverse is correctly creating the schema and attribute names for all the tables/entities.…
asked 2024-06-25T19:12:57.98+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 20%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDO%3C/text%3E%3C/svg%3E)
Dipo Onibile
0
Reputation points
commented 2024-06-28T06:08:57.3866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
7,825
Reputation points Microsoft Vendor
1 answer
CopyData in ForEach
Dear all, I want to use the copydata activity in my pipeline. I have a csv file in my storage account, which has the list of the data i want to pass into the copy activity. I create a lookup activity first and then I try to use ForEach and put the copy…
asked 2024-06-27T09:48:48.74+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 75%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZM%3C/text%3E%3C/svg%3E)
Zhuoqiong Mo
40
Reputation points
commented 2024-06-28T06:02:15.68+00:00
phemanth
7,825
Reputation points Microsoft Vendor
1 answer
High availability of Synapse dedicated SQL pool within a region
We have provisioned a Synapse dedicated SQL pool for our business. We need to make sure that the service is highly available within the region. This means we cannot have the data be moved to another region but need to make sure that the DWH is available…
asked 2024-06-26T03:42:13.2066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 11%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECA%3C/text%3E%3C/svg%3E)
Chand, Anupam SBOBNG-ITA/RX
461
Reputation points
edited a comment 2024-06-28T04:49:08.3866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
Harishga
5,420
Reputation points Microsoft Vendor
0 answers
Unable to use Synapse workspace features in Synapse Link for Dataverse when Synapse network is in private mode
Hi Team, We are using Synapse Link for Dataverse for syncing data from d365 to ADLS Gen2, which can be used for analytics in ADLS Gen2. As our Synapse analytics environment network is publically disabled and using private end points, we cannot enable…
asked 2024-06-25T06:40:07.63+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 15%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
Sriram M
0
Reputation points
commented 2024-06-27T19:29:46.8933333+00:00
BhargavaGunnam-MSFT
28,446
Reputation points Microsoft Employee
0 answers
Azure Synapse analytics with CosmosDB query is giving null values
Hi Team, we are storing data in Azure cosmos db for mongo and trying to query from Azure synapse serverless pool.when we query with Query items with full fidelity schema we are getting the values else we are getting string:value. can you please help us…
asked 2024-06-25T12:09:20.5533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 34%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Sathya Sai Cheetirala
0
Reputation points
commented 2024-06-27T19:29:03.3566667+00:00
BhargavaGunnam-MSFT
28,446
Reputation points Microsoft Employee