pytest in sp creation

Shambhu Rai 1,406 Reputation points
2023-10-12T08:58:27.1166667+00:00

Hi Expert,

have created test case in pytest and wanted to connect to sparksession

how we can add our spark session into it

ModuleNotFoundError: No module named 'pyspark'

getting this error

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,955 questions
Azure App Service
Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
6,983 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Amira Bedhiafi 16,071 Reputation points
    2023-11-24T12:39:42.8566667+00:00

    Maybe you need to install PySpark in your environment where you are running pytest :

    pip install pyspark
    

    To connect to Azure Databricks, you generally need to use the Databricks Connect package.

    pip install databricks-connect
    
    1 person found this answer helpful.
    0 comments No comments