No more allocated vCores on Azure For Students Subscription
I have a Azure For Students subscription, which had 12 allocated vCores for Azure Synapse Workspace, but since yesterday, workspaces only have 0 allocated vCores, making it impossible to run Apache Spark Pools. I have only used 2$ of my 100$ credit and…
No more allocated vCores on Azure For Students Subscription
Hello, I got an azure for students subscription a week ago and Azure Synapse Spark Pool worked just fine with the allocated 12 vCores that come with the subcription. I only used 2$ of credit as of now, but for some reason, since yesterday, new created…
How to check a parquet file version in synapse?
Hi, I tried to copy a parquet file to csv format in gen2 by using copy activity, but failed. It shows the following error: { "errorCode": "2200", "message":…
Adding a non AKV Linked Service does not modify the publish branch in Azure Synapse Analytics
I tried adding a non-AKV based linked service from a feature branch. As per MS Git troubleshoot doc for synapse, it should publish to the azure live synapse service directly, which it did, but it did not create a commit in the publish branch. The doc is…
How to copy all tables of a Schema to .csv file format in respective folders
Hi, I am trying to copy all tables of Schema-SALES from SQL Server to .csv file format in ADF and files of each table must copy in respetive folder for example. source (SQL SERVER Tables) SALES.Customer SALES. Address …
Role Assignment "New-AzRoleAssignment" for Azure Blob Storage is failing using powershell script
Below is my scenario for which I have created a power shell task in Azure DevOps pipeline Scenario= I am trying to query the files from ADLS blob storage in azure synapse workspace. To do this I need to grant "Storage Blob Data Reader" Role…
Data flow sink file name enumeration with LPAD function
Hi, when using a data flow to move data from one container to another and specify the output file names, I am able to get my files enumerated in the style of FileName1.csv, FileName2.csv...but I am not able to get this working with an lpad function.…
Whether the Stream Analytics service supports modifying the row records of the output target database
Hi there, I now have a scenario where I receive data from EventHub through the Azure Stream Analytics service and UPDATE the data into the Azure Synapse table, now I know INSERT is supported, but I don't know if Update is supported. How to update…
automate adding library to spark pool and automate Apache Spark configuration
Our team is trying to do two things 1. automate adding spark-mssql-connector JAR file while the spark pool is creating, not via the UI in synapse workspace automate Apache Spark configuration while the spark pool is creating, not via the UI in…
Using System-Assigned Managed Identity for Blob Storage Access in Azure Synapse Analytics
Hello guys! I'm working on a project in Azure Synapse Analytics and want to switch from using a connection string to a system-assigned managed identity for accessing Blob Storage. I’ve enabled the managed identity and assigned the appropriate role, but…
COPY Activity shows longer duration but its actually less. Which is true?
Hi @Anonymous I have noticed in one of our COPY Activity which shows start date and end date duration as 3 min 30 s but when I dig down deep inside for an activity which took 2 minutes but its actually just 19 seconds where does the remaining 1 min 41…
How to Use System-Assigned Managed Identity for Secure Access to Blob Storage in Synapse Notebook?
Hello Community, I'm currently working on a project in Azure Synapse Analytics and would like to enhance the security of my data access. My Python code in the Synapse notebook currently accesses Blob Storage using a connection string, but I want to…
Why does a COPY Activity shows that it took long time in duration, but when we check the sub-tasks it has actually taken less time
Hi Team, I have a scenario here where a COPY Activity shows that it took long time as per start and end time of the pipeline, but when I check the sub-tasks it has actually taken less time than it is being shown as time taken. Which is true here? If the…
How to use Synapse pipeline to send alert email using Python notebook, but hide username and password to mail server.
I have been asked to create a notebook that can be included in a Synapse pipeline to send alert emails. The examples I have seen use the mail server credentials in the python script. How do I hide these? Server is Outlook. Thanks in advance
Integration runtime is failing
Hi Team, we are unable to run the pipeline using Microsoft Integration Runtime and see following errors. |DEBUG: System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. at …
Notebook : AccessDeniedException
Hello, I am trying to save a delta/parquet file into the ADLS Gen2 using a PySpark Notebook as follows : delta_output_path =…
Automating activity management in synapse data warehouse ETL
We are currently building a Data Warehouse on Synapse using ETL processes. We’re looking for a way to automate the stopping and starting of activities in the event of issues or during deployments, as the current manual method of enabling and disabling…
How to back up and restore the storage account Gen2 of the azure synapse workspace
Dear Team, We are using the "Azure synapse workspace" for D365FO. Our user want to back up and restore the data of the Azure synapse workpace. Please show us how to do this.
Synapse Activity Error but not activity errors
I received activity error alerts at these times I navigated to my synapse analytics metrics to view the activity errors, but there aren't any within this timeframe. The alert is set to trigger and count activities when the activity runs ended and not…
Copy Dataverse data into Azure SQL using Synapse Link
Im using Azure Synapse pipeline to copy data from Dataverse to Azure SQL DB. For large tables, it is only copying 10 million records and it is not inserting after. I have about 15 mil records. I dont see any settings limitation anywhere on the pipeline.…