4,768 questions with Azure Synapse Analytics tags

Sort by: Updated
2 answers One of the answers was accepted by the question author.

ADF in Azure Synapse Analytics Workspace: Trigger Now gives error message "Sync and publish your changes before clicking "Trigger now"."

We have a simple ADF developed in Azure Synapse Analytics Workspace. Running it via "Debug" works fine. Running it via "Trigger Now" gives the error message Sync and publish your changes before clicking "Trigger now". …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,389 questions
asked 2021-01-26T13:00:22.25+00:00
Herven Stefan (SEIT) 41 Reputation points
commented 2021-02-03T17:39:42.33+00:00
Marcos Germano 1 Reputation point
1 answer One of the answers was accepted by the question author.

Synapse spark logs

Hello, Is there a possibility to get access to Spark logs in Synapse other than through Spark UI? I've tried specifying "properties.defaultSparkLogFolder" when creating spark pools but it does not seem to have any effect - what is it used…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-02-01T11:30:24.683+00:00
Irene 71 Reputation points
commented 2021-02-03T15:30:08.7+00:00
Irene 71 Reputation points
0 answers

Synapse query failed after 6 minutes from Data factory

Hi, I'm using the copy activity to extract data from Synapse analytics (SQL DWH). After exactly 6:38 minutes the query fails and the error is a Timeout Error. If a run the query from ssms, works. I set up the max time for query timeout: …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,389 questions
asked 2021-01-25T12:31:30.273+00:00
Jhonatan Reyes 61 Reputation points
commented 2021-02-02T17:27:22.957+00:00
HimanshuSinha-msft 19,466 Reputation points Microsoft Employee
1 answer One of the answers was accepted by the question author.

Synapse workspace - Error code: ClientIpAddressNotAuthorized Inner error code: ClientIpAddressNotAuthorized

Getting this error when trying to save a notebook in Synapse workspace Error code: ClientIpAddressNotAuthorized Inner error code: ClientIpAddressNotAuthorized I can access synapse DB from SSMS and from browser based - Query editor. Any…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-02-02T08:40:16.013+00:00
xRajeev Karn 21 Reputation points
accepted 2021-02-02T11:59:41.55+00:00
xRajeev Karn 21 Reputation points
1 answer One of the answers was accepted by the question author.

Synapse Studio Access control - "Add" button disables

Hello, I'm trying to add a service principal as a Synapse Contributor, but when I go to Manage -> Access Control in Synapse Studio "Add" button is disabled. The user I'm logged in with is Synapse Administrator. Are there any additional…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-01-22T17:01:56.777+00:00
Irene 71 Reputation points
commented 2021-01-29T10:31:49.743+00:00
Shah 1 Reputation point
1 answer

query to specific partition in synapse?

In oracle, to query the particular partition we use following partition keyword i.e SELECT * FROM employees PARTITION (p1); is there any similar command in azure synapse, to query the particular partition in partition table? Also how to identify in…

Azure SQL Database
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-01-18T17:15:43.817+00:00
ash b 1 Reputation point
commented 2021-01-29T07:02:43.993+00:00
HarithaMaddi-MSFT 10,136 Reputation points
1 answer One of the answers was accepted by the question author.

Update-AzSynapseSqlPool suspend or resume not working

Hi, I have installed the Az.Synapse module Install-Module -Name Az.Synapse I have connection to the Synapse Workspace and sqlpool Test-AzSynapseSqlPool -WorkspaceName "synws-futurefdw-devtest" -Name sqlpool True Now I want to…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-01-28T06:52:46.853+00:00
Herven Stefan (SEIT) 41 Reputation points
accepted 2021-01-28T10:00:32.08+00:00
Herven Stefan (SEIT) 41 Reputation points
1 answer One of the answers was accepted by the question author.

Solutioning without ADLS

I am planning to ingest source tables without using ADLS. I do not have any file analytics requirement, just intend to load the source to Synapse and top of that to build dimensional model. Do you see any disadvantage/risk here as I am planning to…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,445 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-01-22T12:24:29.163+00:00
xyz_chatter 211 Reputation points
commented 2021-01-27T09:47:50.293+00:00
xyz_chatter 211 Reputation points
1 answer One of the answers was accepted by the question author.

Error Code "6001"

I am running a Synapse Notebook through a synapse pipeline and am getting an error code 6001 "Creating new Synapse Notebook Statement Failed" and haven't been able to find anything explaining why. When running the code manually, it runs…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-01-26T14:19:02.79+00:00
Al W 96 Reputation points
accepted 2021-01-26T18:40:45.59+00:00
Al W 96 Reputation points
1 answer

Does TR support between On-premise SQL Server and Azure Synapse #5925

Does Microsoft support Transactional Replication between On-premise SQL Server and Azure Synapse?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-01-19T12:18:39.553+00:00
Sarvesh Pandey 141 Reputation points
commented 2021-01-26T18:11:28.927+00:00
Saurabh Sharma 23,796 Reputation points Microsoft Employee
0 answers

urgent - synapse partition table no performance difference?

one of synapse table we've 300 million rows and keep increasing. Every row as status column i.e active_row either 0 or 1. Active_row is int datatype. Users only query based active_row = 1 which has only 28 million row and rest of data i.e 270 million is…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-01-18T17:00:28.667+00:00
ash b 1 Reputation point
commented 2021-01-25T09:22:03.707+00:00
HarithaMaddi-MSFT 10,136 Reputation points
1 answer

ADF load empty strings as null using Polybase

{ "name": " ", "properties": { "activities": [ { "name": " ", "type": "Copy", "dependsOn": [], "policy": { "timeout":…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,389 questions
asked 2021-01-19T08:33:42.207+00:00
FionaHu 1 Reputation point
commented 2021-01-22T18:02:30.04+00:00
MartinJaffer-MSFT 26,071 Reputation points
1 answer One of the answers was accepted by the question author.

Is it possible to replicate data from an Azure SQL Managed Instance database to an Azure Synapse database?

We are considering using Azure SQL managed instance for our OLTP applications and Azure Synapse for analytical applications. For some operational reporting use cases, we need copy of data from OLTP databases inside Azure Synapse. For these scenarios, …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-01-13T16:18:30.437+00:00
Prashanth Padebettu 21 Reputation points
accepted 2021-01-21T14:41:24.287+00:00
Prashanth Padebettu 21 Reputation points
1 answer One of the answers was accepted by the question author.

User with contributor role on Resource group can't query on Synapse Studio

Hello, ...and thanks in advance. I've just joined some users to a resource group that includes a Synapse workspace. I've added them as contributors in the whole resource group and they can access to the Synapse Studio but they has no access to…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-01-18T09:19:33.267+00:00
Daniel MG 21 Reputation points
accepted 2021-01-20T08:09:51.1+00:00
Daniel MG 21 Reputation points
1 answer

How to convert LOTS and LOTS of log data stored in JSON format in Azure Data Lake (Gen2) to parquet files?

I have tens of thousands of devices in the field that store their logs to a centralized Azure Data Lake Gen 2 BLOB storage. The data is JSON format. The data is actually very flat, not a lot of nested arrays, etc. It could almost be a CSV format just…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,116 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,389 questions
asked 2021-01-16T05:06:51.797+00:00
JasonW-5564 161 Reputation points
commented 2021-01-20T07:07:54.887+00:00
HarithaMaddi-MSFT 10,136 Reputation points
1 answer

Fastest way to load data from SQL server to Azure synapse

what's the fastest way to load data from SQL Server to Azure synapse .... suppose the SQL Server is on-prem... Data factory , using SQL Server as Source and Azure synapse as sink will give a best performance ?? or need to SQL Server -> to a…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
SQL Server
SQL Server
A family of Microsoft relational database management and analysis systems for e-commerce, line-of-business, and data warehousing solutions.
13,498 questions
asked 2021-01-18T02:14:26.54+00:00
sakuraime 2,321 Reputation points
commented 2021-01-20T06:59:53.237+00:00
HarithaMaddi-MSFT 10,136 Reputation points
1 answer

How to add .so file to a sys.path directory?

Hi, I would like to add a .so file to one of the sys.path directories. I have attempted to use shutil to copy over from adls, but shutil doesn't recognise this path. Any ideas? Thanks

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-01-11T11:47:54.697+00:00
Ben Holden 1 Reputation point
answered 2021-01-20T06:10:20.647+00:00
MartinJaffer-MSFT 26,071 Reputation points
0 answers

Access Azure Storage account via Synapse through Managed Identity

I am trying to connect to Azure Blob storage via Azure synapse through Managed Identity based on the below set of steps: 1) Assigned an Identity to the Server 2) Gave access to the Server on Blob storage as contributor 3) Executed the below queries …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2020-12-31T09:36:57.91+00:00
Nandan Hegde 31,771 Reputation points MVP
commented 2021-01-19T23:51:57.773+00:00
Saurabh Sharma 23,796 Reputation points Microsoft Employee
2 answers

How to migrate Dedicatel SQL Pool (DW) to Serveless Synapse?

I have Azure Datawarehouse which has fixed pricing tier DW100. Usage is small and assuming that Serverless can be cheaper. A) How do get information how much data processing have been occururing in last month? B) How to do migration to Serverless?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-01-04T10:18:44.87+00:00
Kenny 1 Reputation point
commented 2021-01-19T19:34:21+00:00
HimanshuSinha-msft 19,466 Reputation points Microsoft Employee
2 answers

Error creating table from spark "The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwx-wx---"

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,768 questions
asked 2021-01-11T16:34:32.613+00:00
Richard Luis Díaz 21 Reputation points
answered 2021-01-16T09:43:02.74+00:00
Richard Luis Díaz 21 Reputation points