Where are jar files from Synapse stored so I can access it in Synapse pipelines?

Yuntao 1 Reputation point
2021-11-19T20:00:18.12+00:00

So I currently have a Data Factory pipeline which runs a Databricks job that references a jar file in the Databricks file system (DBFS). So if I wanted to be able to do the same thing in Synapse pipelines and Synapse spark notebook, where would/should the jar file be stored for me to run it in a Synapse pipeline? Basically trying to find the equivalent options of calling the location of the jar file when only using Synapse.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,635 questions
{count} votes

1 answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA-MSFT 53,361 Reputation points Microsoft Employee
    2021-11-24T11:01:14.457+00:00

    Hello @Yuntao ,

    In Azure Synapse Analytics, you can use the Synapse notebook activity to pass in the jar.

    • Upload your jar to the workspace default ADLS gen2 account
    • Then use %%configure in your notebook cell.

    To specify session-scoped Java or Scala packages, you can use the %%configure option:

    %%configure -f  
    {  
        "conf": {  
            "spark.jars": "abfss://<<file system>>@<<storage account>.dfs.core.windows.net/<<path to JAR file>>",  
        }  
    }  
    

    For more details, refer to Azure Synapse Analytics - Session-scoped libraries.

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators