Hello Krishnamohan Nadimpalli ,
Welcome to the MS Q&A platform.
One option I can think of is using Azure Databricks or Synapse notebook.
Using databricks:
You will need to create a data bricks workspace, cluster, and then import your notebook into the workspace.
Install the required python packages into your cluster. Once the packages are installed, you can execute your python code in the notebook via the adf pipeline.
Using synapse:
Create an Azure Synapse workspace and create a dedicated SQL pool or serverless SQL pool.
Launch Azure Synapse Studio and create a new Notebook.
Import your Python notebook into Synapse Studio.
Install the necessary Python packages in your Synapse workspace.
Once the required packages are installed, you can execute your Python notebook via adf notebook activity.
I hope this helps. Please let me know if you have any further questions.