102 questions with Azure Data Lake Analytics tags

Sort by: Updated
1 answer

How to Use System-Assigned Managed Identity for Secure Access to Blob Storage in Synapse Notebook?

Hello Community, I'm currently working on a project in Azure Synapse Analytics and would like to enhance the security of my data access. My Python code in the Synapse notebook currently accesses Blob Storage using a connection string, but I want to…

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
3,216 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,919 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,984 questions
Azure Data Lake Analytics
Azure App Service
Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
7,894 questions
asked 2024-10-28T11:53:36.3733333+00:00
Vaibhav Patil 235 Reputation points
commented 2024-10-30T20:30:00.6366667+00:00
Chandra Boorla 2,605 Reputation points Microsoft Vendor
1 answer

After using api to purge data from workspace the status stays on pending and logs are not removed.

After using api to purge data from workspace the status stays on pending and logs are not removed. While using the status endpoint the status keeps returning as 'pending'. The purge request is visible in activity logs of the workspace but just shows…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,984 questions
Azure Data Lake Analytics
asked 2024-10-21T12:21:41.26+00:00
Danny Baatje 0 Reputation points
commented 2024-10-23T19:15:43.11+00:00
Chandra Boorla 2,605 Reputation points Microsoft Vendor
1 answer One of the answers was accepted by the question author.

Why is the activity log stored in the adls gen2 container not a valid json

Why is the activity log stored in the adls gen2 container not a valid json? It's line by line, and the outer layer is missing '[]' and comma separation

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,480 questions
Azure Data Lake Analytics
asked 2024-10-15T07:08:22.52+00:00
zmsoft 60 Reputation points
accepted 2024-10-16T04:22:46.73+00:00
zmsoft 60 Reputation points
1 answer One of the answers was accepted by the question author.

How does Azure integrate with IoT devices for real-time data processing?

I’m interested in understanding how Azure integrates with IoT devices to enable real-time data processing. What services and tools does Azure offer to process, store, and analyze IoT data streams efficiently? Any examples of how businesses are using…

Azure IoT Hub
Azure IoT Hub
An Azure service that enables bidirectional communication between internet of things (IoT) devices and applications.
1,199 questions
Azure Data Lake Analytics
asked 2024-09-20T17:47:10.7933333+00:00
Mofazzel Hosen Morshed 95 Reputation points
accepted 2024-09-21T09:51:59.7466667+00:00
Mofazzel Hosen Morshed 95 Reputation points
0 answers

I got this error when debug my copy data on azure df 7cf2da46-b65b-4098-b8ec-d9c7858c190e

after connecting my Azure data factory to MsSqL, I tried to copy my data into azure data factory and I got this error 7cf2da46-b65b-4098-b8ec-d9c7858c190e

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,480 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,819 questions
Azure Data Lake Analytics
Azure Data Studio
Azure Data Studio
A cross-platform database tool for data professionals using on-premises and cloud data platforms on Windows, macOS, and Linux.
118 questions
asked 2024-09-13T14:44:21.2833333+00:00
Charles Onyebuchi 20 Reputation points
commented 2024-09-17T15:42:37.1166667+00:00
NIKHILA NETHIKUNTA 2,950 Reputation points Microsoft Vendor
1 answer One of the answers was accepted by the question author.

can we add multiple containers in Microsoft Fabric Data Lake house?

I want to add one more container in Microsoft fabric lake house, as per my knowledge we can add only one container but i am curious to know can we add multiple containers in Microsoft fabric lake house

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,480 questions
Azure Data Lake Analytics
Microsoft Fabric Training
Microsoft Fabric Training
Microsoft Fabric: A Microsoft unified data platform.Training: Instruction to develop new skills.
48 questions
asked 2024-08-26T09:48:53.9066667+00:00
Gnana Sai krishna 80 Reputation points
accepted 2024-08-27T16:00:34.8633333+00:00
Gnana Sai krishna 80 Reputation points
1 answer One of the answers was accepted by the question author.

How to optimize real-time Power BI dashboard through Azure

Hi, I have a Power BI dashboard that displays real-time API traffic data which is working well so far. The Concern is, as the data is stored in an SQL database and connected to BI directly and as database grows, the querying and display times are getting…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,819 questions
Azure Data Lake Analytics
asked 2024-08-06T06:48:58.76+00:00
Jakku Jacob 25 Reputation points
commented 2024-08-06T15:27:24.6066667+00:00
Jakku Jacob 25 Reputation points
1 answer

How to send the history of 3 months of files from one directory to another according to the date in its name

Hello, I'm having problems because I want to move the history of some files stored in a data lake according to the date in its name, but in the end what I manage to do is only move either one specific day or all of them. example: I have files October -…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,480 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,819 questions
Azure Data Lake Analytics
asked 2024-05-14T22:35:22.23+00:00
ALER 0 Reputation points
answered 2024-05-15T09:53:51.7833333+00:00
phemanth 11,125 Reputation points Microsoft Vendor
2 answers

hii want to run stored procedure in synapse but ima getting an error ,can anyone help me

query: USE gold_db GO CREATE OR ALTER PROC CreateSQLServerlessView_gold @ViewName nvarchar(100) AS BEGIN DECLARE @statement VARCHAR(MAX) SET @statement = N'CREATE OR ALTER VIEW '+@ViewName+' AS SELECT * FROM OPENROWSET(BULK…

Azure SQL Database
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
3,216 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,984 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,819 questions
Azure Data Lake Analytics
asked 2024-04-18T16:53:48.23+00:00
naveena kankipati 0 Reputation points
accepted 2024-04-22T10:57:21.55+00:00
naveena kankipati 0 Reputation points
2 answers

Azure Data Lake Gen2 Disaster Recovery/Storage account failover Capabilities?

I am trying to wrap my brain around Azure Data Lake Gen 2 DR/Storage account failover capabilities and I feel as though I am getting conflicting information. On the Microsoft Documentation -…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,480 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
3,216 questions
Azure Data Lake Analytics
asked 2020-06-05T16:28:28.463+00:00
Adam Sebetich 1 Reputation point
commented 2023-12-12T13:47:27.41+00:00
2 answers

remove chars from integer columns in databricks delta

Hi Expert, how to remove chars from column in databricks delta when datatype is not null col1 1 2 3 er ge e

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,480 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,984 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,212 questions
Azure Data Lake Analytics
asked 2023-12-08T12:02:08.32+00:00
Shambhu Rai 1,411 Reputation points
commented 2023-12-10T17:34:15.5333333+00:00
Shambhu Rai 1,411 Reputation points
1 answer

Load files in appending in databricks

Hi Expert, I have 2 files in blob storage . file1.csv, file2.csv how will i load it in a view using appending order using *.csv extension using temp view file_list = dbutils.fs.ls("/mnt/mount/test/file")

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
530 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,212 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,819 questions
Azure Data Lake Analytics
asked 2023-11-29T05:48:38.4166667+00:00
Shambhu Rai 1,411 Reputation points
commented 2023-11-30T12:32:49.06+00:00
Shambhu Rai 1,411 Reputation points
1 answer

jdfc url of blob storage

Hi , i wanted to push the files to blob storgate using portman ... how to connect using jdbc connection i tried below but unable to push it.. can i get jdbc file for…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,480 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,212 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,819 questions
Azure Data Lake Analytics
asked 2023-11-06T12:32:08.7566667+00:00
Shambhu Rai 1,411 Reputation points
commented 2023-11-15T07:05:29.57+00:00
PRADEEPCHEEKATLA-MSFT 90,146 Reputation points Microsoft Employee
0 answers

Datalake account_partitioned table row count is verly low

We have azure datalake with builtin pool In that we have builtin account_Partitioned table which use to have similar no or rows as per non partitioned table but now from today only showing 10% of total rows We are in UK West region and something…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,480 questions
Azure Data Lake Analytics
asked 2023-11-06T18:07:39.4833333+00:00
Musarrat Aziz 5 Reputation points
edited the question 2023-11-15T07:01:44.1266667+00:00
PRADEEPCHEEKATLA-MSFT 90,146 Reputation points Microsoft Employee
1 answer

Correction on Microsoft PL-300 Practice Assessment Questions

This is to bring to the notice of the Admin in charge of Practice Test Questions under Pl-300 Power Bi Exam. There is a confusion in one of the questions and answers. Please find attached snap shot and correct the answers Thank you

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
530 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,212 questions
Azure Data Lake Analytics
Windows 365 Business
asked 2023-10-24T10:51:44.65+00:00
Ayotomide Ogunsakin 0 Reputation points
edited the question 2023-11-06T07:46:50.6233333+00:00
Sumarigo-MSFT 46,441 Reputation points Microsoft Employee
1 answer One of the answers was accepted by the question author.

Data lake zones and containers, what is the best practices of file name

Hi All, follow up Best practices from cloud Adoption framework, can somebody tell standers for file Name Data LakeLayersContainer numberContainer nameFile Name Standard1Raw1Landing?1Raw2Conformance?2Enriched1Standardized?2Curated2Data…

Azure Data Lake Analytics
asked 2023-10-16T07:58:21.98+00:00
manish verma 461 Reputation points
accepted 2023-10-20T13:10:33.5533333+00:00
manish verma 461 Reputation points
1 answer

Can we pass partition key as a parameter in a generic/dynamic azure dataflow.

Operation on target Load Entity failed: Operation on target Load Entity failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: com.microsoft.dataflow.Issues: DF-TX-114 - Key partitioning does…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,984 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,819 questions
Azure Data Lake Analytics
asked 2023-10-05T08:43:06.7+00:00
Batta, Uha Kanakavalli 0 Reputation points
commented 2023-10-12T18:26:57.77+00:00
QuantumCache 20,271 Reputation points
1 answer One of the answers was accepted by the question author.

Help me decide between Azure data lake analytics and Azure Databricks

Both Azure Data Lake Analytics and Azure Databricks can be used for batch processing. Can you help me understand for which case we can use each one?

Azure Analysis Services
Azure Analysis Services
An Azure service that provides an enterprise-grade analytics engine.
456 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,212 questions
Azure Data Lake Analytics
asked 2023-10-04T10:40:23.2033333+00:00
WeirdMan 260 Reputation points
accepted 2023-10-04T14:55:59.32+00:00
WeirdMan 260 Reputation points
2 answers One of the answers was accepted by the question author.

RowsWritten from data flow in log analytics

I was recently able to get the rowsCopied and rowsRead from a pipeline copy activity in log analytics....I'm trying to get the rowsWritten from the pipeline dataflow activity but I'm not getting any input. Is this even logged in log analytics? and if…

Azure Monitor
Azure Monitor
An Azure service that is used to collect, analyze, and act on telemetry data from Azure and on-premises environments.
3,313 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,819 questions
Azure Data Lake Analytics
asked 2021-09-28T07:13:57.18+00:00
Adeleke Abimbola 231 Reputation points
commented 2023-09-11T15:15:13.5833333+00:00
Pratibha 0 Reputation points
1 answer

ADF converts NUMBER datatype to Decimal, String (if precision > 28) while copying data type from ORACLE to ADLS as Parquet

While extracting data from Oracle on premise to Azure Data Lake as Parquet files via SHIR, we notice the below points: NUMBER Datatype in Oracle gets converted to Decimal, String (if precision > 28) in parquet file Kindly share some light on the…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,480 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
3,216 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,919 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,819 questions
Azure Data Lake Analytics
asked 2023-07-31T04:53:47.64+00:00
Swapnil Sarkar 0 Reputation points
commented 2023-08-19T07:30:36.56+00:00
AnnuKumari-MSFT 33,316 Reputation points Microsoft Employee