Can't access txt file with json format from Data Lake Gen2 inside a Data Flow
I have stored a txt file with json format (coming from a MongoDB) in my Data Lake Storage Gen2 and inside my Data Flow I would like to access that file, convert it into a column structure, transform the data and shift it to my Azure SQL DB. Currently I…
Common data service is not able to connect to Azure data lake store Gen 2 when "Selected Networks" is configured in firewall.
Issue: Common data service is not able to connect to Azure data lake store Gen 2 when "Selected Networks" is configured in firewall. Description: Both environment and storage account are in west europe region. However, the reason connection…
Data labeling and classification solution in ADLS gen2
Hi, I need to classify and the label with the classification the data stored in Data Lake Storage e.g. assign Sensitive label on some folder/file and then have a way to present in some report, provide retention based on the classification etc. One…
USQL Datalake Question
Is there anyway to backup the partition files of usql table? Or the only way is to import the data to csv file.
Import incremental data from Azure Data Lake
Good afternoon, I have the following scenario !! I have a table in SQL that always receives changes daily, and I need to send this data to Azure data lake So, today I take this data and send it in the datalake file: Historicos.csv …
Unable to load data into Azure Synapse from ADLS Gen 2 .csv file
Hi @ChiragMishraMSFT-1092 Upon trying to load the data from ADLS(csv files) into Azure DW, am encountering a error in loading the data in DW. Here is the error message: "message": "Failure happened on 'Sink' side.…
Express route and Azure Data Lake gen 2
Hi, I am doing some testing around accessing Azure Data Lake Gen 2 from an On Premise hadhoop I need to access the service via Express Route What is the recommended setup if anyone has been through a configuration? thanks
How do I create an exploratory zone on top of my Azure Data Lake
Hi, My sincere apologies if this question would have been already answered. However, I searched hard but couldn't find the relevant information. Here lies my question: I have a Data Lake setup in Azure already. Further on, I have a requirement…
Write json document to azure table
Hi, I am using below code to write json document in a Azure Data Lake Gen2 container into a SQL Server table. Code: df =…
ADL Gen2 rename using Path REST API failure with SAS token
Hi, I am currently trying to rename a file within a container using the https://learn.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/path/create API and an account-level SAS token. The SAS token was generated from the Azure portal…
What is the recommended approach if we had to load data from an Azure Data Lake Gen 2 to Oracle Exadata?
What is the recommended approach if we had to load data from an Azure Data Lake Gen 2 to Oracle Exadata?
How to Transform files in subfolders with one script in databricks
i have a adls gen2 folder with sub folders with parquet files in each folder. My requirement is to transform all parquet files in sub folders and load into another folder in adls gen 2 with same folder structure with one script. is it possible to do or…
ADLS Gen2 Upload issue within IBM Java 8 in AIX machine
Hi Team, We have deployed a web application in AIX machine with IBM Java 8. In this web application, we are invoking ADLS Gen2 Rest API to upload and download the files. We are getting the following error, when we try to upload the files to ADLS Gen2…
Ad integration pass through for HDInsight
HI, I know HDInsight with ESP feature enables AD integration while connecting to cluster. Also teh access to underlying Hive tables can be controlled using Apache Ranger. But i would like to know if the access permission on storage account or datalake…
How to set ACL to folder and all its children recursively?
I'm using the following code to set a ACL to a folder and its children in Azure Storage Account StorageV2. We have more than 2 million files and this takes too long. I could't find any recursive parameter or a best aproach. Is this the best option to…
import json payload from a rest api and save as json documents in adls gen2
Hi, I am trying to import json payload from a REST api GET method and save json documents into ADLS Gen2 using azure databricks. GET: https://myapi.com/api/v1/city GET method Output: [ {"id":2643743, …
How to improve throughput of COPY activity
Hello, Please advise me to improve the throughput of data copy from SAP. ![15331-image.png][1] [1]: /api/attachments/15331-image.png?platform=QnA The same number of rows copy from another SAP server(different SHIR), but that has high throughput. Your…
Access Azure DataLake G2 from Azure subscription
Hello I 'm deploying an ETL scenario to move data from SQL server to Azure DataLake G2. When starting the process, I can see that specifying Azure DataLake G2 as target is defining Azure Blob Storage as target. Is this the right procedure ? …
Upgrade Access Tier via Azure CLI or Python Code for Stoarge Account Gen2
We Want to update the access tiers in the ADLS Gen2 for Multiple paths and want to use Azure CLI or Python Code as per our requirement. According to Microsoft Documentation, We see only Portal and Power shell code to do it. Can anyone let us know if…
FileNotFoundException when using abfss to list files in Azure Databricks!
Hi team, I am trying to connect to ADLS2 using hadoop configurations: But when I am trying to use FS commands to list all the files on the path, i am getting File not found exception: import org.apache.hadoop.fs.{FileSystem, Path} …