Databricks : to Merge files in Blob storage

Rohit Kulkarni 731 Reputation points
2022-12-08T06:34:52.627+00:00

Hello Team,

I need to merge the 7 days old files into 1 files in azure blob storage from the databricks.Which is the best solution. please advise. how it can be done.

Regards
Rohit

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
{count} votes

1 answer

Sort by: Most helpful
  1. KranthiPakala-MSFT 46,737 Reputation points Microsoft Employee Moderator
    2022-12-12T01:56:36.853+00:00

    Hello @Rohit Kulkarni ,

    Thanks for the question and using MS Q&A platform.

    Below are couple of stack overflow threads where a similar requirement has been discussed by community volunteers using pyspark in Databricks. You can follow similar approach to achieve your requirement.

    1. Merge CSV files in ADLS2 that are prepared through DataBricks 269451-image.png
    2. Azure Data-bricks : How to read part files and save it as one file to blob?

    Hope this helps. Let us know how it goes.

    0 comments No comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.