output spark job definition (activity) result to next activity

alexander tikhomirov 31 Reputation points

I know there is a documented feature to output result from synapse notebook activity to use in next activity, but is this possible for spark job definition activity?



Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
3,791 questions
{count} votes

1 answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA-MSFT 69,606 Reputation points Microsoft Employee

    Hello @alexander tikhomirov ,

    Thanks for the question and using MS Q&A platform.

    Unfortunately, in Spark job definition activity doesn't populates the output, so that can be utilized for the next activities.

    Note: By default, the Spark Job definition activity writes output to the path which you have passed in the command line arguments:


    Since you already knows the output folder of the Spark job definition. Once the spark job definition is successfully, you can directly parameterize the output path in the next activity.


    Hope this will help. Please let us know if any further queries.

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators