Fabric Native Execution
Sriram Ganarajan
0
Reputation points
Hi ,
Is the native execution avaialble for free trails? I signed up for a free trial , but when i had attempted the Fabric Native Execuiton, The launch of Spark App failed as below
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:757)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.shuffle.sort.ColumnarShuffleManager
My Spark launc prop as below
%%configure
{
"conf": {
"spark.native.enabled": "true",
"spark.shuffle.manager": "org.apache.spark.shuffle.sort.ColumnarShuffleManager"
}
}
The way intepreted this, is the additional jars needed for this shuffle may not be in the classpath load. Which made me suspect if the Fabic Spark Pool i attemped is having the needed feature to support .
To test that , i had separately executed the property
"spark.native.enabled": "true",
I could not notice any Changes to Spark Plan and it was as before. Anything am missing here?
Sign in to answer