Retirement of Apache Spark 3.1

Question

Received an e-mail about the retirement of Apache Spark 3.1 for the Azure Synapse runtime. We don't have any defined Apache Spark pools, the only thing we have is an Azure Integration runtime that data flows utilize. I'm a little confused as to where we "upgrade" to Spark 3.3.

Answer 1

@Jeremy D. Busselman - Thanks for the question and using MS Q&A platform.

It is possible that you received the retirement notice email because you have a subscription that includes Azure Synapse Analytics, even if you are not currently using it. The email may have been sent to all subscribers who have access to Azure Synapse Analytics, regardless of whether they are actively using it or not.

Scroll down to find the account information details in your subscription

To confirm whether you have any resources that need to be upgraded, you can check your Azure subscription for any resources related to Azure Synapse Analytics or Apache Spark. You can also check your Azure billing history to see if you have been charged for any usage of these services.

If you do have resources that need to be upgraded, you can follow the guidance provided in the retirement notice email to transition to newer version as Azure Synapse runtime for Apache Spark 3.1 will be retired on 26 January 2024 – Transition to Apache Spark 3.3 or newer to ensure they continue to run as usual. If you have code that's incompatible with the version you transition to, you can troubleshoot it using the guidance provided in the email.

Note: If you do not have any resources that need to be upgraded, you can safely ignore the retirement notice email.

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.