An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
Hi Christine Han,
Yes, your overall approach is correct. Using Azure PowerShell to centrally inventory Spark pool versions across all Azure Synapse workspaces is the right method to confirm compliance before the Spark 3.4 end-of-support deadline (March 31, 2026).
However, to make this fully accurate and enterprise-ready, a few important refinements are required.
First, if your 50 workspaces span multiple subscriptions, the script must iterate across all subscriptions. Running it in a single subscription context may miss non-compliant Spark pools.
Second, version comparison should not rely on string comparison (for example, "3.10" would incorrectly evaluate as less than "3.5"). Versions should be cast to [version] for reliable comparison.
Below is the finalized, production-ready script:
Connect-AzAccount
$results = @()
$subscriptions = Get-AzSubscription
foreach ($sub in $subscriptions) {
Set-AzContext -SubscriptionId $sub.Id
$workspaces = Get-AzSynapseWorkspace
foreach ($ws in $workspaces) {
$pools = Get-AzSynapseSparkPool -WorkspaceName $ws.Name -ResourceGroupName $ws.ResourceGroupName
foreach ($pool in $pools) {
$isCompliant = ([version]$pool.SparkVersion -ge [version]"3.5")
$results += [PSCustomObject]@{
Subscription = $sub.Name
Workspace = $ws.Name
ResourceGroup = $ws.ResourceGroupName
SparkPool = $pool.Name
SparkVersion = $pool.SparkVersion
ComplianceStatus = if ($isCompliant) { "Compliant" } else { "Needs Upgrade" }
}
}
}
}
$results | Export-Csv -Path "SparkStatus.csv" -NoTypeInformation
This ensures: All subscriptions are evaluated Accurate version comparison Centralized CSV output for audit and governance review
Regarding requirements, Spark 3.4 reaches end of support on March 31, 2026. Upgrading to Spark 3.5 GA (or later supported runtime) ensures continued security patches, bug fixes, and platform support. After upgrading, validate custom libraries, JAR/Wheel dependencies, Delta Lake compatibility, and any notebook or job runtime references.
Regarding Service Tree and S360 tracking: Azure Support does not have direct visibility into internal Service Tree IDs for runtime validation. Compliance must be verified at the Azure resource level. Spark runtime lifecycle upgrades are platform announcements and are not automatically tracked as S360 remediation items unless your organization has integrated Azure resource compliance signals into S360 governance workflows. If you have not received S360 items, this is expected behav
Your validation strategy is correct. With the refinements above, it will provide a complete tenant-wide compliance confirmation before the March deadline.