how to join databricks table with sqldb table in databricks

Yu, Hazel (APEX SYSTEMS LLC) 6 Reputation points
2022-04-04T20:53:43.027+00:00

Hi
I'm trying to join a table from sqldb with the databricks table to fill out a column with null values in sqldb table.
more details are in the question link below.

https://stackoverflow.com/questions/71741886/how-to-join-databricks-table-with-sqldb-table-in-databricks

I would really appreciate any help to accomplish this.. Thank you

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,076 questions
{count} votes

1 answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA-MSFT 85,511 Reputation points Microsoft Employee
    2022-04-06T10:23:21.157+00:00

    Hello @Yu, Hazel (APEX SYSTEMS LLC) ,

    Thanks for the question and using MS Q&A platform.

    As per the repro from my end, I had created two dataframes name df1 => table1 and df2 = table2 as per your requirement.

    190535-image.png

    How to join databricks tables with sqldb table.

    As shown above read both tables into the dataframe and apply inner join as shown below:

    df = df1.join(df2, df2.label_name == df1.label_name, "inner").select(df2.label_name,df2.unit)  
    display(df)  
    

    190541-image.png

    Now you can convert dataframe into a table in the SQLDB as shown:

    190543-image.png

    You can verify the same from SQL endpoint:

    190536-image.png

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators