How to define multiple main methods in a Jar class (in Scala) and call it from Azure Data Factory?

Question

Hi,

I am looking to create Jar file class which contains multiple main class methods(in Scala) so Azure data factory activity can call jar activity. Azure data factory requires main class name and parameters.

Regards Rajaniesh

Answer 1

@Rajaniesh, Welcome to the Microsoft Q&A platform.

Jar files can contain only one Main-Class attribute in the manifest, which means a jar can contain only one mainClassName.

Here is the sample JSON definition of a Databricks Jar Activity:

{  
    "name": "ADB-Jar-Activity",  
    "properties": {  
        "activities": [  
            {  
                "name": "Jar1",  
                "type": "DatabricksSparkJar",  
                "dependsOn": [],  
                "policy": {  
                    "timeout": "7.00:00:00",  
                    "retry": 0,  
                    "retryIntervalInSeconds": 30,  
                    "secureOutput": false,  
                    "secureInput": false  
                },  
                "userProperties": [],  
                "typeProperties": {  
                    "mainClassName": "org.apache.spark.examples.SparkPi",  
                    "parameters": [  
                        "10",  
                        "20"  
                    ],  
                    "libraries": [  
                        {  
                            "jar": "dbfs:/docs/sparkpi.jar"  
                        }  
                    ]  
                },  
                "linkedServiceName": {  
                    "referenceName": "ADB",  
                    "type": "LinkedServiceReference"  
                }  
            }  
        ],  
        "annotations": []  
    }  
}  

Reference: Transform data by running a Jar activity in Azure Databricks

Hope this helps. Do let us know if you any further queries.

----------------------------------------------------------------------------------------

Do click on "Accept Answer" and Upvote on the post that helps you, this can be beneficial to other community members.