Edit

Kongsi melalui

Azure Synapse Runtime for Apache Spark 3.2 (deprecated)

  • Artikel

Azure Synapse Analytics supports multiple runtimes for Apache Spark. This document covers the runtime components and versions for the Azure Synapse Runtime for Apache Spark 3.2.

Caution

Deprecation and disablement notification for Azure Synapse Runtime for Apache Spark 3.2

  • End of Support announced for Azure Synapse Runtime for Apache Spark 3.2 July 8, 2023.
  • Effective July 8, 2024, Azure Synapse will discontinue official support for Spark 3.2 Runtimes.
  • In accordance with the Synapse runtime for Apache Spark lifecycle policy, Azure Synapse runtime for Apache Spark 3.2 will be retired as of July 8, 2024. After the End of Support date, the retired runtimes are unavailable for new Spark pools and existing workflows can't execute. Metadata will temporarily remain in the Synapse workspace.
  • We strongly recommend that you upgrade your Apache Spark 3.2 workloads to Azure Synapse Runtime for Apache Spark 3.4 (GA) before July 8, 2024.

Component versions

Component Version
Apache Spark 3.2.1
Operating System Ubuntu 18.04
Java 1.8.0_282
Scala 2.12.15
Hadoop 3.3.1
.NET Core 3.1
.NET 2.0.0
Delta Lake 1.2
Python 3.8
R (Preview) 4.2

Synapse-Python38-CPU.yml contains the list of libraries shipped in the default Python 3.8 environment in Azure Synapse Spark.

Libraries

To check the libraries included in Azure Synapse Runtime for Apache Spark 3.2 for Java/Scala, Python and R go to Azure Synapse Runtime for Apache Spark 3.2 Release Notes

Next steps

Migration between Apache Spark versions - support

For guidance on migrating from older runtime versions to Azure Synapse Runtime for Apache Spark 3.3 or 3.4, refer to Runtime for Apache Spark Overview.