US government regulations with azure
I am looking for detailed documentation on ( Family Educational Rights and Privacy Act) FERPA and I am looking for useful information. Still, I could not get a detailed implementation of FERPA with various Azure services such as ADLS, Azure Databricks,…
Terabytes of data to Azure
I am required to ingest 50+TB of data from a learning system with all types of structured, unstructured, and semi-structured data. Since the data volume is large, I am not sure the methods below are the right tools to ingest 50+TB of data into ADLS …
API source data and intermediatory storage
I am moving data from a highly secure system with compliance as report as service (RaaS) as I discussed in the questions below.…
can we add multiple containers in Microsoft Fabric Data Lake house?
I want to add one more container in Microsoft fabric lake house, as per my knowledge we can add only one container but i am curious to know can we add multiple containers in Microsoft fabric lake house
Setting different permissions for SAS Token on ADLS Gen2
Hello, I'm interested to know if it's possible to set two different sets of permissions for an access policy on ADLS Gen2 Storage - one at the container level, and another at the folder level. I'm looking for a setup like this: Access Policy setting…
Cost of ADLS Gen2 while enabling Az synpase link for Cosmos database
Hi Team, I am exploring the Azure synapse link for Azure cosmos database, but when I enable the Azure synapse link then it require the azure synapse workspace to be created along with Azure data lake storage Gen2. So, below are my queries around it. …
Multiple Storage Accounts for a Data lake
All, We are thinking of implementing the Zones using Storage Accounts. Which means, each of the 3 Zones (RAW, Staged, Curated) will have one Storage Account each. Are there any best practice document? Is it normal to have more than one Storage…
How to get parquet file data from data lake gen2 in tabular form using postman or web api.
i have a file in my blob storage gen2 container i want to fetch this file data by using api or postman in the tabular form by using query without azure synapse, azure sql and databricks.
Can't connect to Microsoft Search Graph Connector - ADLS Gen2
We are trying to add a new “ADLS Gen2” Connector in Microsoft Search and we are receiving this error below when testing the connection. We are sure that we provided the correct Connection String from access keys within our storage account in Azure. We’ve…
Read Delta Tables from ADLS Gen2 in Power BI: Exploring Direct Access Methods
As part of our data pipeline, I'm using Azure Blob Storage as the source, Azure Data Factory (ADF) for processing, and Azure Data Lake Storage Gen2 (ADLS Gen2) as the destination. Here's the process I've followed: Step 1: I read a CSV file from Azure…
Got error code 403 when try to access my Synapse workspace created within a virtual network
I got a 403 error when access my Synapse Workspace created in a managed virtual network within a jumphost inside my virtual network. This is the network settings of my Synapse Analytics workspace My bicep files have help me set up some private…
![](https://techprofile.blob.core.windows.net/images/UVyLn-Xlr0apNce5TRX1RA.png?8DBA3B)
Data size of databricks delta tables
It has been observed that the size of delta tables are much less as compared to when checked the underlying delta files in the storage account. Suppose a databricks delta table raw.deltaTableA has size of 2MB if we check the size of underlying delta…
Replicating a Pipeline Multiple Times in Azure Synapse Analytics
How can I replicate a pipeline multiple times in Azure Synapse Analytics? I need to replicate my pipeline 10 times with only a change to the table name which is passed as a parameter. Thanks in advance!
In the context of optimizing data workflows using Microsoft Power BI, how would you leverage the DAX (Data Analysis Expressions) language to enhance performance for complex aggregations and calculations in large datasets?
In the context of optimizing data workflows using Microsoft Power BI, how would you leverage the DAX (Data Analysis Expressions) language to enhance performance for complex aggregations and calculations in large datasets? Discuss the techniques for…
Storage Datalake allows upload, but a read causes a 401
I'm following the documentation here: https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-directory-file-acl-javascript Have verified, triple verified the account name and the account key. Uploading a file works: async function…
How to access ADLS Gen2 storage with restricted network access from PySpark notebook
I'm trying to access ADLS Gen2 storage from PySpark notebook in my Synapse workspace. The storage account has public network access enabled from selected virtual networks and IP addresses, and my IP address is added to the firewall rules, trusted Azure…
How to send a mail notification for a failed pipeline in Azure Synapse Analytics?
How can I send a notification email to a specific email address without using a logic app when one of my Synapse Analytics pipelines fails? I would like to include the error message in the email notification.
Business Central Connection with Azure Data Factory
Hello, is there a way to connect Business Central to Azure Data Lake? I am only seeing one for NAV/AX and D365 CRM Apps.
Cannot turn on Updata and Process on Premium storage SAS
I create a premium storage account and try to add Shared access signature. In the allowed permissions, I cannot turn on "Update" and "Process" selection. See Screenshot 2024-08-13 at 2.48.42 PM.jpg. But I have no problem to turn…