4,362 questions with Azure Synapse Analytics tags

Sort by: Updated
1 answer

Moving multiple SQL server data warehouse databases from On premise to Azure

Dear Experts, I am new to Azure. I have few questions please help me. we have one SQL server dw database per each customer on premise for reporting (using SSRS). Each dw database consists few dimension tables, one fact and one staging table. …

Azure Analysis Services
Azure Analysis Services
An Azure service that provides an enterprise-grade analytics engine.
437 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
asked 2022-06-24T21:21:02.96+00:00
Sreenivasa Chary Tatikonda 1 Reputation point
commented 2022-08-01T18:55:44.33+00:00
HimanshuSinha-msft 19,376 Reputation points Microsoft Employee
1 answer One of the answers was accepted by the question author.

Can't write Pandas df to ADLS from Synapse Notebook - No such file or directory exists.

I'm using mount point in synapse notebook as shown in MS Azure Docs. I'm using same mount point and same path to read and write spark dataframe and that is working fine but while operation with pandas I'm facing this issue. import pandas as pd …

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,338 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,526 questions
asked 2022-07-26T04:32:54.44+00:00
Amit Sopan Shendge 26 Reputation points
accepted 2022-08-01T15:06:06.2+00:00
Amit Sopan Shendge 26 Reputation points
1 answer

How do I add a new string to the folder name? - Azure Synapse Analytics

I would like to add a new string to the Blob folder name when bulk copying data from S3 to Azure Blob Storage. The folder structure of S3 is shown in the following picture Folder Name/yyyy/mm/dd/files. I would like to copy it to Blob…

Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,427 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,526 questions
asked 2022-07-07T06:57:08.23+00:00
Kakehi Shunya (筧 隼弥) 201 Reputation points
commented 2022-08-01T09:57:28.517+00:00
ShaikMaheer-MSFT 37,896 Reputation points Microsoft Employee
1 answer

I would like to add the date and time string in addition to the source file name.

When copying a file from S3 to AzureBlobStorage, I would like to add the date and time string in addition to the source file name. In essence, the S3 folder structure looks like this data/yyyy/mm/dd/files *yyyy=2019-2022, mm=01-12, dd=01-31 …

Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,427 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,526 questions
asked 2022-07-07T14:28:40.487+00:00
Shunya Kakehi 1 Reputation point
commented 2022-08-01T09:56:04.26+00:00
ShaikMaheer-MSFT 37,896 Reputation points Microsoft Employee
0 answers

Synapse Link for Dataverse can't be accessed in pipelines

Hi, we setup the link between dataverse and synapse, sync our table. fine, working. I can run SQL statements in Azure Synapse. but as soon as I try to connect a pipeline activity, like a simple copy, I can't. I have this error: …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
asked 2022-07-20T01:12:30.887+00:00
Jérôme Po 1 Reputation point
commented 2022-08-01T09:44:42.543+00:00
ShaikMaheer-MSFT 37,896 Reputation points Microsoft Employee
1 answer One of the answers was accepted by the question author.

Issue sinking from Azure Data Factory v2 to Azure Synapse Warehouse with Greek letters

We are trying to import data that contains Greek letters into Azure Synapse Analytics (Data Warehouse) via Azure Data Factory v2. We are getting replacement characters for the Greek characters. Collation for Data Warehouse:…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,526 questions
asked 2022-07-26T23:01:00.773+00:00
Cullen, Cory 21 Reputation points
commented 2022-08-01T09:14:50.817+00:00
ShaikMaheer-MSFT 37,896 Reputation points Microsoft Employee
0 answers

Create synapse credential for power bi

Hi, I wish to create a az synapse analytics credential for power bi that has a scheduled load

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
asked 2022-07-28T08:12:14.037+00:00
Alexandra Varadi 1 Reputation point
commented 2022-08-01T08:41:31.48+00:00
ShaikMaheer-MSFT 37,896 Reputation points Microsoft Employee
2 answers One of the answers was accepted by the question author.

BlobServiceClient timeout

Problem: we are reaching a timeout error with the BlobServiceClient instead of receving a error that shows that the credentials are invalid: from azure.storage.blob import BlobServiceClient …

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,683 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
asked 2022-07-15T14:39:43.523+00:00
Lorenz Weinzierl 21 Reputation points
accepted 2022-08-01T07:33:08.96+00:00
Lorenz Weinzierl 21 Reputation points
1 answer One of the answers was accepted by the question author.

Is there a way to see how expression gets evalated in "if condition"

Hi, I have a very basic doubt. Suppose i have an If condition activity in my pipeline. It has an expression that gets evaluated to a boolean value. I want to see how the expression is interpreted or evaluated in run time, in other words whats…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,526 questions
asked 2022-07-19T05:18:47.827+00:00
Suba Balaji 11,186 Reputation points
accepted 2022-08-01T06:10:30.83+00:00
Suba Balaji 11,186 Reputation points
1 answer

How to Flatten A Complex Nested JSON Structure in Synapse

As the title suggests, I'm trying to take a complex JSON file, flatten it, and put it into a dedicated SQL pool table. The data is an output from the Shopify API orders endpoint. Here is a sample, it is one single order: [ { …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,526 questions
asked 2022-07-19T01:54:30.367+00:00
Bill LaLonde 11 Reputation points
commented 2022-08-01T06:00:10.637+00:00
AnnuKumari-MSFT 30,601 Reputation points Microsoft Employee
1 answer

what-is-the-synapse lake database -security-setup-to-allow-reader

I have a requirement where i have to give read access to the user in Azure Data Lake Synapse , is there any way by which we can provide read access to the user ? As of now I have observed that we need to provide user admin access to the synapse…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
asked 2022-07-13T12:22:21.19+00:00
Ankita Gupta 1 Reputation point
commented 2022-08-01T05:42:26.017+00:00
AnnuKumari-MSFT 30,601 Reputation points Microsoft Employee
2 answers One of the answers was accepted by the question author.

Using XML in Synapse spark pool

Hi, Can anybody help with using the XML in Synapse spark pool with pyspark? I found some articles where they suggest a code like this would load the XML into a data frame, but I get an error when trying it: …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
asked 2022-07-27T14:10:39.577+00:00
Ljubo Jurkovic 41 Reputation points
accepted 2022-07-28T15:12:14.543+00:00
Ljubo Jurkovic 41 Reputation points
1 answer

Failing select command of external statement

Msg 105019, Level 16, State 1, Line 12 External file access failed due to internal error: 'Error occurred while accessing HDFS: Java exception raised on call to HdfsBridge_IsDirExist. Java exception message: HdfsBridge::isDirExist - Unexpected…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
asked 2022-06-27T07:26:59.487+00:00
Kothapalli Srinivasarao 131 Reputation points
commented 2022-07-28T10:56:57.987+00:00
Raul Hosu 1 Reputation point
1 answer One of the answers was accepted by the question author.

Dedicated SQL Pool - Can I import a sql server table or a sql server database into azure Synapse dedicated sql pool?

I have existing sql server tables. I would like to archive them in Azure Synapse Dedicated pool as it is said that one can use T-sql to query in Azure Synapse. My question is: Can I import sql server tables into Azure Synapse ( structure and…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
asked 2022-07-26T18:44:46.467+00:00
Nguyen, Hoa 331 Reputation points
accepted 2022-07-27T13:31:36.427+00:00
Nguyen, Hoa 331 Reputation points
1 answer

Azure Synapse Analytics: Publishing error Error while publishing: Cannot read properties of undefined (reading 'constructor')

I tried to: -delete last changes -remove git and reenable it

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
asked 2022-07-20T15:01:40.257+00:00
Domenica Lozada Guerra 1 Reputation point Microsoft Employee
commented 2022-07-27T04:40:21.807+00:00
BhargavaGunnam-MSFT 25,976 Reputation points Microsoft Employee
1 answer

How to refresh data in Synapse Lake Database using pipeline DataFlow

I have a pipeline set up in Synapse which runs 2 dataflow activities which is supposed to 1. copy data from Fhir as source to Azure SQL Server as sink and 2. copy data from Azure SQL Server as source to Azure Lake as sink. First, is this the best…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
asked 2022-07-14T17:15:58.21+00:00
TerriblyVexed 51 Reputation points
commented 2022-07-27T02:54:31.42+00:00
BhargavaGunnam-MSFT 25,976 Reputation points Microsoft Employee
2 answers One of the answers was accepted by the question author.

synapse spark pool - pyspark load a subset of xml files from given folder

I am limited to use 4 vCores / 32 GB - 3 to 10 nodes configuration of apache spark pool. I am trying to load all xml files from given folder with below code: …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
asked 2021-07-18T07:13:52.333+00:00
Dheeraj 351 Reputation points
answered 2022-07-26T20:59:43.153+00:00
Ljubo Jurkovic 41 Reputation points
3 answers One of the answers was accepted by the question author.

Flowlets question

Hi, I am exploring the usage of flowlets and wonder how I should implement the following scenario: I have a dataflow which reads a source with 10 columns, one of these columns contains a customer number for which I want a flowlet to execute some logic on…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,526 questions
asked 2022-03-11T07:57:14.743+00:00
Marc van der Wielen 46 Reputation points
commented 2022-07-26T18:21:27.273+00:00
John Neubecker 81 Reputation points
0 answers

Data trasnfer across teants using flows

Hi All, I want to copy data from Kusto(MSFT tenant) and ingest to ADLS Gen2(PME tenant). How to do it securely? Can I do it without using App Registration?

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
480 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
asked 2022-07-26T07:11:20.95+00:00
kk 41 Reputation points
commented 2022-07-26T09:11:15.703+00:00
kk 41 Reputation points
1 answer

LinkedService in Deleting state but not actually being deleted

I'm running the following command via a Azure Powershell job in my Release Pipeline in Azure DevOps: Install-Module Az.Synapse -RequiredVersion 0.7.0 -Scope CurrentUser -Force Remove-AzSynapseLinkedService -WorkspaceName…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,362 questions
asked 2022-07-11T07:24:30.363+00:00
Victor Seifert 151 Reputation points
commented 2022-07-26T06:37:34.547+00:00
Victor Seifert 151 Reputation points