Hi @Upul Karunaratne , Thank you for posting query on Microsoft Q&A Platform.
Yes, you can use Azure Powershell cmdlets to upgrade Apache Spark pool from 3.1 to 3.3.
Update-AzSynapseSparkPool - Updates a Apache Spark pool in Azure Synapse Analytics.
As per repro, I was able to upgrade Apache Spark pool from 3.1 to 3.3 using Update-AzSynapseSparkPool powershell cmdlet as shown below. Regarding your second question, if you don't have any defined Spark pools, you don't need to upgrade them. However, if you are running Apache Spark Application batch jobs, which are Synapse Pipelines triggered daily, you should upgrade the Spark version to 3.3 to avoid any issues.
Regarding your third question, existing pipelines and dataflows should work after upgrading to Spark version 3.3. However, it's always a good practice to test your pipelines and dataflows after upgrading to a new version of Spark to ensure everything is working as expected.
Finally, regarding the automatic upgrade of Spark version to 3.3 on January 26, 2024, Microsoft will handle the upgrade process, and you don't need to do anything. However, it's always a good practice to upgrade to the latest version of Spark to take advantage of new features and bug fixes.
Hope this helps. Do let us know if you any further queries.
If this answers your query, do click Accept Answer
and Yes
for was this answer helpful. And, if you have any further query do let us know.