azure file share map drive with AAD authentication without joining the domain?
through Storage key i am able to map the azure file share drive but instead of storage access key authentication how to enable azure file share mapping through AAD authentication and our laptops are workstation not AD joined? Also while creating the…
Error at create spark database in synapse workspace notebook using orchestrate pipeline.
I'm getting below error while running synapse notebook from orchestrate in synapse workbook. My notebook runs fine when ran individually from Develop. I gave Owner and Storage Blob Data Owner role to myself before running the pipeline as mentioned in…
Azure Data Lake Gen2 Disaster Recovery in case of failover , we dont find an option for Manual Failover?
I am planning for ADLS gen 2 storage with GA-GRS option to have replicated data from one to another region. DR is capable for Azure Data Lake Storage Gen 2, its just an automatic process that Microsoft kicks off in the event the storage account…
Azure Data Lake Gen2 Snapshots and point in time recovery isnt available?
I would like to enable snapshots or point in time recovery for azure Data Lake Store Gen2 but don't find any option for it. How do we restore data in case of accidental deletion by user and i would like to test it in my environment once.
$log container Name not in ARM template of azure Storage but it is deplyo on Target
HI All, i generate a ARM template for azure storage account- in JSON i didn't find any $log container, but after deployment i found a $log container. I Try to delete this container $log but not able to delete. message- i don't have permission. i…
Appending an empty file from local machine to ADLSGen2 Storage's file.
Does it make sense to append an empty file to an existing file in Azure Data Lake Storage Gen2? I have a csv file in Azure Data Lake Storage Gen2, I'm trying to append data to the same via REST API Call. In one of the scenario the data to be…
Data lake storage Gen 2 timeout issue
Facing timeout issue if we access list of paths in a selected file system. Using azure-storage-file-datalake 12.1.2 jdk 11. fileSystemClient.listPaths().forEach(path -> System.out.printf("Name: %s%n", path.getName())); Working fine…
Data migration to Azure data lake using sas failes suddenly
HI , i have a process which is copying data to azure data lake using sas key. from 27th june 2020 around 4:00 am ust it stopeed suddenly giving error message "java.util.NoSuchElementException: An error occurred while enumerating the result, check…
What is the best way to query data from Azure Data Lake ?
I am ingesting all variety of data in Azure Data Lake Gen 2(Azure Blob Storage with Hierarchical namespace enabled)? What is the best way to index data and to query data from Data Lake. Data Ingested can be in form of CSV, Json, XML files.
Uploading files from OneDrive to Azure Data Lake Gen2
This is the situation. A client of my has XML files coming from Peoplesoft. I would like the client to place them in a inbox on a OneDrive location. With Azure Logic Apps i want to create a trigger that sends an emial when there is a new file placed in…
java.io.IOException: Stream is closed! Error in HDInsight with ADLS Gen 2
I am currently using Hail for the pyspark library to perform varying operations on Genomic data in ADLS Gen 2 with an HDInsight 4.0, Spark 2.4 cluster. I have been in touch with the development team regarding this error I have been getting when running a…
Can't process ORC files in Data Factory : ErrorCode=ParquetJavaInvocationException
Hi, Our organisation uses ORC formatted files for our central file storage, in our data factory I am unable to process most of the ORC files, I am only able to process really small files (< 1mb) for all the other ORC files the pipeline fails to run. …
How to pull the data from Dynamic 365 to Data lake
<p>Hi <a rel="user" nodeId="74270" href="/answers/users/74270/chiragmishramsft-1092.html">@ChiragMishraMSFT-1092</a> </p> <p>Can you pls help me with the below error:</p> <p>{…
Azure Data Lake storage file directory listing Json Serialization Error
Hi, I am having troubles in listing the directories in in the Azure Data Lake storage. I am pretty much using the default template for listing the directories on a file system according to data-lake-storage-directory-file-acl-dotnet . I have wrapped…
how to replicate ongoing changes of on prem database to data lake?
Like AWS DMS handles ongoing replication and update bucket with respective files, what is available in azure to manage ongoing changes occurring at on-prem sql database? Initially it requires to do a full load in my data lake storage which can be…
Extract the date hierarchy from previous data in Data Lake Partition
Hi Team, Would be of great help to get some clear assistance on the below query. Data source is Dynamic365 & SAP Hana. My requirement is to create a data lake partition(ADLS Gen2) with date…
Can a new user assign to Posix style ACL's without RBAC in Data Lake storge gen2
Hi all, i have a active directory user say A i want to assign him Posix style ACL's- folder permission without RABC permission. it is possible or not please confirm
how can we create a custom role in Data Lake Store Gen 2 has access on azure storage user can view folder and container but not read file data
HI All, in Data Lake Gen2. we have one requirement- we need to create a custom role-adding user in this role User- should be able to view folder , but not able to read data. how can we create a custom role to get this requirement
Access Control Exception on setting data lake storage gen1 as output
Hi, I am receiving JSON data as input of stream analytics job from IOT hub and trying to write it in Data Lake Storage Gen-1. The stream analytics job has read, write and execute permissions and I have read permission to Data Lake storage. According to…
More convenient service to read avro files from Azure Data Lake Gen2
Hi, I have to read lots of avro files created by an Event Hub Capture in a Data Lake Gen2. Data must be filtered, processed and then applied to train a machine learning model. I'm considering Azure Databricks and the Azure Machine Learning service…