Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,416 questions
1 answer
One of the answers was accepted by the question author.
テナントを跨いだPowerBIの活用について
弊社にはazureのテナントが二つあります。 １のテナントにはPowerBIProライセンスがあり、2つ目のテナントのDateLake内にはPowerBIで可視化させたいデータがあります。 2つ目のデータを可視化させるためにオンプレミスデータゲートウェイを活用すれば1つ目のテナントにあるPowerBIを使用できますでしょうか。 宜しくお願い致します。 Convert Japanese to English: Title: Utilization of Power BI across…
asked 2020-11-09T06:19:11.863+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 65%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EWS%3C/text%3E%3C/svg%3E)
Watanabe Shoya (渡邉 頌也)
21
Reputation points
accepted 2020-11-09T07:55:08.653+00:00
Watanabe Shoya (渡邉 頌也)
21
Reputation points
1 answer
One of the answers was accepted by the question author.
Use ADF to load a file in TEAMS into ADL
Is there a way to load an xlsx file in teams into Azure Data Lake using Azure Data Factory? Thanks geert
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,416 questions
Microsoft Teams
Microsoft Teams
A Microsoft customizable chat-based workspace.
9,542 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,074 questions
asked 2020-11-05T17:56:04.897+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 37%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGV%3C/text%3E%3C/svg%3E)
geert vanhove
116
Reputation points
accepted 2020-11-06T09:14:21.173+00:00
geert vanhove
116
Reputation points
0 answers
Azure Data Lake Gen2 - Use Case Advice
I am collecting weather data (history and forecast) from a third part web service. Since there will be a lot of data, and it will not have high use, I was planning to use Azure Data Lake Gen2 with blob storage, and storing the data in JSON files. My…
asked 2020-10-30T14:58:46.347+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 27%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEM%3C/text%3E%3C/svg%3E)
Eric Maibach
1
Reputation point
commented 2020-11-05T18:02:15.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 26%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HimanshuSinha-msft
19,381
Reputation points Microsoft Employee
2 answers
Error while loading data from Azure Data Lake Gen2 to Azure Synapse Analytics using Azure Data Factory
Hi All, I am getting an error while loading data from Azure Data Lake Gen2 to Azure Synapse Analytics using Azure Data Factory. I am unable what exactly the error is? What permission has to be provided? How?. This error occurs when I use Polybase…
asked 2020-08-16T09:00:27.563+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 55.00000000000001%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EI%3C/text%3E%3C/svg%3E)
ImranMaqsood
6
Reputation points
answered 2020-11-04T04:57:11.47+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 77%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERH%3C/text%3E%3C/svg%3E)
Rudolph Harms
26
Reputation points
1 answer
Data Factory Copy activity failing with Error code 2200 - Operation failed as split count exceeding upper bound of 1000000
Hi , I am copying large data from ADLS gen 1 to ASDW(Azure synapse Analytics) using Data Factory (polybase set to yes and mentioned blob storage settings). The Source data is *.parquet format and is partitioned. Copy activity fails. In first attempt…
asked 2020-10-25T13:53:15.673+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 93%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKK%3C/text%3E%3C/svg%3E)
kranthi k senapathi
106
Reputation points
commented 2020-10-29T17:45:16.183+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 9%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HarithaMaddi-MSFT
10,136
Reputation points
1 answer
HDInsight azure adls gen2 'InternalServerError' ARM Template deployment
Creating Azure HDinsight Spark cluster with ADLS Gen 2,Userassigned managed idnetity with StorageBlobdataOwner role. Successfully assigned msi role to storage but getting error with HDInsight deployment(Internal server error) Theres some issue with…
asked 2020-10-08T04:56:53.52+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 8%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anon4914
1
Reputation point
commented 2020-10-28T07:51:37.547+00:00
PRADEEPCHEEKATLA-MSFT
84,856
Reputation points Microsoft Employee
2 answers
One of the answers was accepted by the question author.
Getting size of each folders in containers in adls gen2
Hi, I need to get length of folders and files of Adls gen2 using powershell .. my requirement is ilke -- i will pass container name then it should recursively should show length of files and all folders and it should iterate through all folders…
asked 2020-10-27T05:23:07.98+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 28.000000000000004%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERB%3C/text%3E%3C/svg%3E)
Rohit Boddu
461
Reputation points
commented 2020-10-28T05:28:31.68+00:00
Rohit Boddu
461
Reputation points
1 answer
One of the answers was accepted by the question author.
How to get list of child files/directories having parent DataLakeDirectoryClient class instance
Hello All! I have to scan whole data lake file system. Having code like: PagedIterable<PathItem> pItems = ((DataLakeFileSystemClient)prmParent).listPaths(); for( PathItem pItem : pItems ){ if pItem.isDirectory() ){ …
asked 2020-10-22T10:48:28.173+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 1%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Sergey Shabalov
46
Reputation points
accepted 2020-10-25T16:46:51.523+00:00
Sergey Shabalov
46
Reputation points
1 answer
One of the answers was accepted by the question author.
Unable to add service principle, groups to the $logs container in adls2
Recently enabled storage analytics on ADLS Gen2 storage account.I can see the $logs container and the logs are writing to this on an hourly basis. But when I'm trying to add service principal to this container getting permission denied. I'm able to…
asked 2020-10-09T17:21:44.103+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 62%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPH%3C/text%3E%3C/svg%3E)
Praneeth H
26
Reputation points
accepted 2020-10-22T21:41:43.803+00:00
Praneeth H
26
Reputation points
2 answers
Data Masking in Azure data Factory
We are using Azure Data factory to move data from Source like Azure SQL and Azure Postgres to destination as Azure data lake.There is some sensitive data which needs to be masked. Is it possible to have data masking in Azure Data factory during…
asked 2020-10-18T07:24:11.37+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 15%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHG%3C/text%3E%3C/svg%3E)
Himanshu Garg
1
Reputation point
answered 2020-10-21T01:29:56.217+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 17%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
Kiran-MSFT
691
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Getting Data from azure datalake through api
Hi Team, Looking for any api available to get data from azure data lake directly ,with out using any data transformation tools like pipelines/dataflow. Csv or Json files sitting in datalake storage folders, so the same data or files can be fetched…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 80%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPK%3C/text%3E%3C/svg%3E)
0 answers
My partitions are returning inconsistent results (Mapped JSON file data)
I have a set of tables based on a number of JSON files taken from a data lake v2 container. I'm building an external table using derived data from JSON mapping, then using derived columns based on those columns to generate an identifier. When I go to…
asked 2020-10-09T21:00:12.793+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(233.6, 71%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EME%3C/text%3E%3C/svg%3E)
Mark Ehlke
1
Reputation point
commented 2020-10-15T17:19:26.137+00:00
HimanshuSinha-msft
19,381
Reputation points Microsoft Employee
1 answer
Azure Data Lake Gen2 PUT authorization
I'm trying to create a Shared Access Signature client side in my Node app. The reason being that I do not want to stream files through my app. I want the user to be able to upload a file to my Azure Data Lake Gen2 Blob Storage container directly. I have…
asked 2020-09-25T07:22:12.537+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(19.2, 13%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMV%3C/text%3E%3C/svg%3E)
Magnus Vhendin
1
Reputation point
commented 2020-10-13T23:28:51.437+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 70%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
MartinJaffer-MSFT
26,051
Reputation points
1 answer
One of the answers was accepted by the question author.
Writing parquet file throws…An HTTP header that's mandatory for this request is not specified
I have two ADLSv2 storage accounts, both are hierarchical namespace enabled. In my Python Notebook, I'm reading a CSV file from one storage account and writing as parquet file in another storage, after some enrichment. I am getting below error when…
asked 2020-10-12T23:48:19.033+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 33%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPM%3C/text%3E%3C/svg%3E)
Porsche Me
131
Reputation points
accepted 2020-10-13T13:44:57.697+00:00
Porsche Me
131
Reputation points
1 answer
Where to store delta-files
Hey guys! I have been wondering about the below question for a while and I hope you can help me get a good night sleep again. Question: When you are working in Azure DataBricks to do transformations, you can save the results using a delta format.…
asked 2020-10-05T13:05:54.297+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 21%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECG%3C/text%3E%3C/svg%3E)
Christian Gert Hansen
1
Reputation point
commented 2020-10-08T20:15:56.767+00:00
HimanshuSinha-msft
19,381
Reputation points Microsoft Employee
1 answer
Error Setting Access Rights on ADL (Gen1) User Folders During HDInsight Cluster Creation
Hello All The issue occurs when the cluster is created. One of the last operations that Ambari performs are 'Post user creation hook for 1 users'. This fail with the attached errors. Here is some more information: It's an ESP cluster -- we…
asked 2020-09-22T07:21:55.527+00:00
Christoph Kiefer
141
Reputation points
commented 2020-10-07T13:04:47.837+00:00
PRADEEPCHEEKATLA-MSFT
84,856
Reputation points Microsoft Employee
3 answers
Can't connect to our ADLS gen2 in Powerbi Online (scheduled refresh)
Hi all, We have recently set up a ADLS gen 2 and we want to update our reports automatically. In the powerBI desktop version everything works, but online I get the following error: Failed to update data source credentials: The credentials provided…
asked 2020-07-15T14:41:43.403+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(313.6, 71%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
Joris
6
Reputation points
answered 2020-10-01T17:08:46.117+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 96%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEF%3C/text%3E%3C/svg%3E)
Elizabeth Fohl
1
Reputation point Microsoft Employee
1 answer
Mounting entire ADLS on azure databricks
Hi, I want to mount entire ADLS storage account on databricks. I've checked in documents I can mount a single filesystem at a time but I want to mount entire ADLS on databricks. I've around 70 containers in my ADLS and I want to mount all of them…
asked 2020-09-23T11:35:43.993+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 38%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAG%3C/text%3E%3C/svg%3E)
Ashima Gupta
1
Reputation point
commented 2020-09-29T19:04:09.19+00:00
MartinJaffer-MSFT
26,051
Reputation points
1 answer
One of the answers was accepted by the question author.
Adls Availability Alerts
Hi Team, I have set alerts on my adls gen2 such as if availability is less than 80 percent it should send alert notification. My question is this availability is Microsoft responsibility, right ? how to find out root cause of why this availability…
asked 2020-09-29T05:32:42.44+00:00
Rohit Boddu
461
Reputation points
accepted 2020-09-29T09:05:16.513+00:00
Rohit Boddu
461
Reputation points
1 answer
One of the answers was accepted by the question author.
Copy Different type of file from Gen1 Azure lake to Azure Gen2 lake with attribute( like last updated)
I need to migrate all my data from Azur data lake Gen1 to Lake Gen2. In my lake we have different types of file mixed (.txt, .zip,.json and many other). We want to move them as-it-is to GEN2 lake. Along with that we also want to maintain last updated…
asked 2020-09-20T16:47:33.427+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(249.60000000000002, 62%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBK%3C/text%3E%3C/svg%3E)
Bhuvnesh Kumar
26
Reputation points
commented 2020-09-29T07:10:41.99+00:00
Bhuvnesh Kumar
26
Reputation points