Hello Lakshmi P,
Welcome to the Microsoft Q&A platform.
It seems like the error is related to the capacity of the spark pool.
Please try to scale up your spark pool and see if it resolves.
I hope this helps. Please let us know if you have any further questions.