databrciks-pyspark-sql-fdataframe

arkiboys 9,686 Reputation points
2022-05-06T14:08:49.337+00:00

How is it possible to write sql query against a dataframe in azure databricks?
At present I do filter on dataframe but not sure how to write select against it.

Thank you

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,070 questions
0 comments No comments
{count} votes

Accepted answer
  1. AnnuKumari-MSFT 32,011 Reputation points Microsoft Employee
    2022-05-09T12:15:48.917+00:00

    Hi @arkiboys ,
    Thankyou for using Microsoft Q&A platform and posting your query.
    As I understand your ask, you want to perform SQL queries on Dataframe in Databricks. Please let me know if my understanding is incorrect.

    All sql queries which we want to perform on dataframes can be done from functions available inside pyspark.sql library. For example, as you mentioned for performing WHERE clause of SQL we can make use of filter() function on dataframe

    If you are looking for explicitly writing only SQL queries on dataframe then I will suggest to create a temp view from Dataframe and then write sql on it directly. Please check below screenshot :

    200284-image.png

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you.
      Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
    0 comments No comments

0 additional answers

Sort by: Most helpful