4,974 questions with Azure Synapse Analytics tags

Sort by: Updated
0 answers

No more allocated vCores on Azure For Students Subscription

I have a Azure For Students subscription, which had 12 allocated vCores for Azure Synapse Workspace, but since yesterday, workspaces only have 0 allocated vCores, making it impossible to run Apache Spark Pools. I have only used 2$ of my 100$ credit and…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2024-10-31T20:28:57.7433333+00:00
Ramzi Belbahri 0 Reputation points
0 answers

No more allocated vCores on Azure For Students Subscription

Hello, I got an azure for students subscription a week ago and Azure Synapse Spark Pool worked just fine with the allocated 12 vCores that come with the subcription. I only used 2$ of credit as of now, but for some reason, since yesterday, new created…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2024-10-31T20:23:37.7666667+00:00
Ramzi Belbahri 0 Reputation points
1 answer

How to check a parquet file version in synapse?

Hi, I tried to copy a parquet file to csv format in gen2 by using copy activity, but failed. It shows the following error: { "errorCode": "2200", "message":…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2024-10-30T18:00:06.6333333+00:00
Zhu, Yueli YZ [NC] 235 Reputation points
commented 2024-10-31T20:06:58.46+00:00
Zhu, Yueli YZ [NC] 235 Reputation points
0 answers

Adding a non AKV Linked Service does not modify the publish branch in Azure Synapse Analytics

I tried adding a non-AKV based linked service from a feature branch. As per MS Git troubleshoot doc for synapse, it should publish to the azure live synapse service directly, which it did, but it did not create a commit in the publish branch. The doc is…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2024-10-31T17:31:49.0666667+00:00
Nandu S Raj 0 Reputation points
edited the question 2024-10-31T17:42:43.5466667+00:00
Nandu S Raj 0 Reputation points
2 answers One of the answers was accepted by the question author.

How to copy all tables of a Schema to .csv file format in respective folders

Hi, I am trying to copy all tables of Schema-SALES from SQL Server to .csv file format in ADF and files of each table must copy in respetive folder for example. source (SQL SERVER Tables) SALES.Customer SALES. Address …

Azure SQL Database
Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
5,069 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,800 questions
asked 2023-12-22T21:18:17.5066667+00:00
Zaeemul Haque 25 Reputation points
commented 2024-10-31T15:21:02.04+00:00
Wenrong Xu 0 Reputation points
2 answers

Role Assignment "New-AzRoleAssignment" for Azure Blob Storage is failing using powershell script

Below is my scenario for which I have created a power shell task in Azure DevOps pipeline Scenario= I am trying to query the files from ADLS blob storage in azure synapse workspace. To do this I need to grant "Storage Blob Data Reader" Role…

Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,915 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2022-11-11T05:50:24.527+00:00
sachin gupta 376 Reputation points
answered 2024-10-31T14:13:22.8233333+00:00
Killerbe 50 Reputation points
1 answer

Data flow sink file name enumeration with LPAD function

Hi, when using a data flow to move data from one container to another and specify the output file names, I am able to get my files enumerated in the style of FileName1.csv, FileName2.csv...but I am not able to get this working with an lpad function.…

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
5,069 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,800 questions
asked 2024-10-25T15:25:58.4433333+00:00
ROWE, GENEVIEVE 0 Reputation points
commented 2024-10-31T10:32:18.7633333+00:00
Vinodh247 22,696 Reputation points
1 answer

Whether the Stream Analytics service supports modifying the row records of the output target database

Hi there, I now have a scenario where I receive data from EventHub through the Azure Stream Analytics service and UPDATE the data into the Azure Synapse table, now I know INSERT is supported, but I don't know if Update is supported. How to update…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
359 questions
asked 2024-10-31T06:29:46.07+00:00
zmsoft 40 Reputation points
answered 2024-10-31T10:00:40.05+00:00
Vinodh247 22,696 Reputation points
0 answers

automate adding library to spark pool and automate Apache Spark configuration

Our team is trying to do two things 1. automate adding spark-mssql-connector JAR file while the spark pool is creating, not via the UI in synapse workspace   automate Apache Spark configuration while the spark pool is creating, not via the UI in…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2024-10-29T17:53:24.9833333+00:00
Chelsea Chen 0 Reputation points Microsoft Employee
commented 2024-10-30T20:42:05.6266667+00:00
Chandra Boorla 2,525 Reputation points Microsoft Vendor
1 answer

Using System-Assigned Managed Identity for Blob Storage Access in Azure Synapse Analytics

Hello guys! I'm working on a project in Azure Synapse Analytics and want to switch from using a connection string to a system-assigned managed identity for accessing Blob Storage. I’ve enabled the managed identity and assigned the appropriate role, but…

Microsoft Identity Manager
Microsoft Identity Manager
A family of Microsoft products that manage a user's digital identity using identity synchronization, certificate management, and user provisioning.
706 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,915 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2024-10-28T12:13:39.88+00:00
Vinit Joshi 5 Reputation points
commented 2024-10-30T20:39:27.1666667+00:00
Chandra Boorla 2,525 Reputation points Microsoft Vendor
1 answer

COPY Activity shows longer duration but its actually less. Which is true?

Hi @Anonymous I have noticed in one of our COPY Activity which shows start date and end date duration as 3 min 30 s but when I dig down deep inside for an activity which took 2 minutes but its actually just 19 seconds where does the remaining 1 min 41…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2024-10-28T12:04:39.9233333+00:00
TheNerd-0042 50 Reputation points
commented 2024-10-30T20:38:18.4466667+00:00
Chandra Boorla 2,525 Reputation points Microsoft Vendor
1 answer

How to Use System-Assigned Managed Identity for Secure Access to Blob Storage in Synapse Notebook?

Hello Community, I'm currently working on a project in Azure Synapse Analytics and would like to enhance the security of my data access. My Python code in the Synapse notebook currently accesses Blob Storage using a connection string, but I want to…

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
3,220 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,915 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
Azure Data Lake Analytics
Azure App Service
Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
7,879 questions
asked 2024-10-28T11:53:36.3733333+00:00
Vaibhav Patil 235 Reputation points
commented 2024-10-30T20:30:00.6366667+00:00
Chandra Boorla 2,525 Reputation points Microsoft Vendor
1 answer

Why does a COPY Activity shows that it took long time in duration, but when we check the sub-tasks it has actually taken less time

Hi Team, I have a scenario here where a COPY Activity shows that it took long time as per start and end time of the pipeline, but when I check the sub-tasks it has actually taken less time than it is being shown as time taken. Which is true here? If the…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2024-10-28T11:53:29.73+00:00
TheNerd-0042 50 Reputation points
commented 2024-10-30T20:29:43.12+00:00
Chandra Boorla 2,525 Reputation points Microsoft Vendor
1 answer One of the answers was accepted by the question author.

How to use Synapse pipeline to send alert email using Python notebook, but hide username and password to mail server.

I have been asked to create a notebook that can be included in a Synapse pipeline to send alert emails. The examples I have seen use the mail server credentials in the python script. How do I hide these? Server is Outlook. Thanks in advance

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2024-10-22T14:31:12.97+00:00
NUNZIATA, Frank (NHS RESOLUTION) 20 Reputation points
accepted 2024-10-30T19:07:13.68+00:00
NUNZIATA, Frank (NHS RESOLUTION) 20 Reputation points
1 answer

Integration runtime is failing

Hi Team, we are unable to run the pipeline using Microsoft Integration Runtime and see following errors. |DEBUG: System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. at …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2024-10-30T13:49:45.89+00:00
Riyaz Ahmed Khaja Hussain 0 Reputation points
answered 2024-10-30T16:08:29.5533333+00:00
Jonathan Pereira Castillo 7,700 Reputation points Microsoft Vendor
1 answer

Notebook : AccessDeniedException

Hello, I am trying to save a delta/parquet file into the ADLS Gen2 using a PySpark Notebook as follows : delta_output_path =…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,208 questions
asked 2024-10-29T15:28:46.6966667+00:00
Wajih Arfaoui 70 Reputation points
commented 2024-10-30T13:21:58.4566667+00:00
Smaran Thoomu 16,555 Reputation points Microsoft Vendor
1 answer

Automating activity management in synapse data warehouse ETL

We are currently building a Data Warehouse on Synapse using ETL processes. We’re looking for a way to automate the stopping and starting of activities in the event of issues or during deployments, as the current manual method of enabling and disabling…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2024-10-29T11:27:56.0433333+00:00
Gabriel-2005 285 Reputation points
commented 2024-10-30T13:20:41.2266667+00:00
Smaran Thoomu 16,555 Reputation points Microsoft Vendor
1 answer

How to back up and restore the storage account Gen2 of the azure synapse workspace

Dear Team, We are using the "Azure synapse workspace" for D365FO. Our user want to back up and restore the data of the Azure synapse workpace. Please show us how to do this.

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
3,220 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2024-10-22T02:00:00.7466667+00:00
Phuong Pham 0 Reputation points
commented 2024-10-30T13:11:01.6933333+00:00
Smaran Thoomu 16,555 Reputation points Microsoft Vendor
0 answers

Synapse Activity Error but not activity errors

I received activity error alerts at these times I navigated to my synapse analytics metrics to view the activity errors, but there aren't any within this timeframe. The alert is set to trigger and count activities when the activity runs ended and not…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
asked 2024-10-24T09:07:41.5233333+00:00
Iwan 65 Reputation points
commented 2024-10-30T13:08:31.1+00:00
Smaran Thoomu 16,555 Reputation points Microsoft Vendor
1 answer

Copy Dataverse data into Azure SQL using Synapse Link

Im using Azure Synapse pipeline to copy data from Dataverse to Azure SQL DB. For large tables, it is only copying 10 million records and it is not inserting after. I have about 15 mil records. I dont see any settings limitation anywhere on the pipeline.…

Azure SQL Database
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,478 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,974 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,800 questions
Microsoft Dataverse Training
Microsoft Dataverse Training
Microsoft Dataverse: A Microsoft service that enables secure storage and management of data used by business apps. Previously known as Common Data Service.Training: Instruction to develop new skills.
40 questions
asked 2024-10-23T15:49:18.2433333+00:00
Murali Ramasamy 5 Reputation points
edited a comment 2024-10-30T05:22:00.5933333+00:00
phemanth 11,125 Reputation points Microsoft Vendor