Share via

How to save a Synapse Analitycs query results in a blob storage

GABRIEL SOLON PADILHA 1 Reputation point
2022-10-05T19:01:23.903+00:00

So, I have some parquet files stored in azure devops containers inside a blob storaged. I used data factory pipeline with the "copy data" connector to extract the data from a on premise Oracle database.

I'm using Synapse Analytics to do a sql query that uses some of the parquet files store in the blob container and I want to save the results of the query in another blob. Which Synapse connector can I use the make this happen? To do the query I'm using the "develop" menu inside Synapse Analytics.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,696 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,197 questions
Transact-SQL
Transact-SQL
A Microsoft extension to the ANSI SQL language that includes procedural programming, local variables, and various support functions.
4,601 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. AnnuKumari-MSFT 32,161 Reputation points Microsoft Employee
    2022-10-06T10:07:00.293+00:00

    Hi @GABRIEL SOLON PADILHA ,

    Welcome to Microsoft Q&A platform and thanks for posting your question here.

    As per my understanding , you are trying to load the output of parquet file from one container to another using synapse analytics. Please correct me if my understanding is wrong.

    You can consider using Notebook in synapse analytics where you can write pyspark code to create a dataframe and load the output of parquet file into the same dataframe and then write that dataframe into blob storage in specified file format.

    Note: Kindly make sure that your Azure AD user has read/write permissions on the storage account.

    Kindly check the below code where I am trying to read parquet file and writing it as csv in the storage account.

    248008-image.png

    For more details, kindly check : Read & write parquet files using Apache Spark in Azure Synapse Analytics

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you.
      Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators