Share via

databricks as source

arkiboys 9,711 Reputation points
2022-02-23T09:13:37.847+00:00

Hello,
In ADF I am using databricks activity to connect to a notebook which does a select * from xyz.
Question:
Then how do I land this result into my data storage?
I can use a copy activity but what should the source of the copy activity be having mentioned the above example?
Thank you

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

0 comments

3 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Pratik Somaiya 4,211 Reputation points Volunteer Moderator
    2022-02-23T13:04:54.603+00:00

    Can you try exploring Azure DeltaLake for Databricks

    Was this answer helpful?

  2. arkiboys 9,711 Reputation points
    2022-02-23T12:34:56.283+00:00

    Hi,
    As mentioned before, there databricks view return several fields.
    These are fields which I would like to access from the copy source activity.
    How do I do this?
    Thanks

    Was this answer helpful?

    0 comments
  3. Pratik Somaiya 4,211 Reputation points Volunteer Moderator
    2022-02-23T10:30:30.733+00:00

    You can load the results of your select query in storage by directly connecting your Notebook to Storage

    You can visit below article which explains how to connect to Storage from a Databricks Notebook

    The same notebook which returns select result can store it in Storage as well

    https://www.sqlshack.com/accessing-azure-blob-storage-from-azure-databricks/

    As a first step you will have to mount Azure Storage in Databricks, the steps are added in the link above

    Thanks!

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.