spark.readStream.text() text file from adls gen2 in databricks

Sasidhar R. Kolli 20 Reputation points
2023-10-05T15:48:37.58+00:00
file_path = "adl://<account-name>.dfs.core.windows.net/<container>/<path>/*.txt"

streaming_df = spark.readStream \
    .schema(schema) \
    .text(file_path) 

query = streaming_df_transformed.writeStream \
    .outputMode("append") \
    .format("console") \
    .start()

query.awaitTermination()

this is the code I'm using but when i read simple spark.read.text it is able to read my df but when I do spark.readStream.text() I'm able to read but when I do query it is throwing me error could not find adls gen2 token. Please provide me solution

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,365 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,960 questions
Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
334 questions
{count} votes