This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 75%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Hi Expert,
have created test case in pytest and wanted to connect to sparksession
how we can add our spark session into it
ModuleNotFoundError: No module named 'pyspark'
getting this error
Suggestion pls
@Shambhu Rai - Thanks for the question and using MS Q&A platform.
Could you please share the code snippet which you have tried along with the screenshot of the error message which you are experiencing?
it says launch gateway option is not available
SparkContext._gateway = gateway or launch_gateway(conf)
^^^^^^^^^^^^^^^^^^^^
how we can configure workspace
@Shambhu Rai - Could you please share the code snippet and screenshot which you are referring too?
Here is the complete code i wanted to crate databricks connection and attach notebook instead of this. can you please share the exact step to connect with databricks
from pyspark.sql import SparkSession
from pyspark.sql.types import *
spark = SparkSession.builder.getOrCreate()
schema = StructType([
StructField('CustomerID', IntegerType(), False),
StructField('FirstName', StringType(), False),
StructField('LastName', StringType(), False)
])
data = [
[ 1000, 'Mathijs', 'Oosterhout-Rijntjes' ],
[ 1001, 'Joost', 'van Brunswijk' ],
[ 1002, 'Stan', 'Bokenkamp' ]
]
customers = spark.createDataFrame(data, schema)
customers.show()
# Output:
#
# +----------+---------+-------------------+
# |CustomerID|FirstName| LastName|
# +----------+---------+-------------------+
# | 1000| Mathijs|Oosterhout-Rijntjes|
# | 1001| Joost| van Brunswijk|
# | 1002| Stan| Bokenkamp|
# +----------+---------+-------------------+
suggestion please
suggestion pls
suggestion pls
Suggestion pls
@Shambhu Rai - Thanks for sharing the code which you are using, kindly share the complete stack trace of the error message on which steos it throwing this error message?
Sir there is no error.. How can we pass notebook here
suggestion please
suggestion please
@Shambhu Rai - We are reaching out to the internal team to get more information related to your query and will get back to you as soon as we have an update.
suggestion please
@Shambhu Rai - Please do share the notebook as the text file as an attachement.
sir, thats what i want to know ...have mentioned the sample above...how add notebook with functions
can you please specified your approach on top of my code in terms of ammendment
suggestion please
Maybe you need to install PySpark in your environment where you are running pytest :
pip install pyspark
To connect to Azure Databricks, you generally need to use the Databricks Connect package.
pip install databricks-connect