Databricks Runtime 16.0
The following release notes provide information about Databricks Runtime 16.0, powered by Apache Spark 3.5.0.
Databricks released this version in November 2024.
Tip
To see release notes for Databricks Runtime versions that have reached end-of-support (EoS), see End-of-support Databricks Runtime release notes. The EoS Databricks Runtime versions have been retired and might not be updated.
Behavioral changes
- Breaking change: JDK 17 is now the default
- Breaking change: Hosted RStudio is end-of-life
- Breaking change: Removal of support for changing
byte
,short
,int
andlong
types to wider types - Correct parsing of regex patterns with negation in nested character grouping
- Improve duplicate match detection in Delta Lake
MERGE
- Cluster library installation method can no longer be overridden
- Default timeout of two hours for cluster-scoped library installs
- Installing libraries from DBFS and setting the spark conf
spark.databricks.driver.dbfsLibraryInstallationAllowed
are disabled - The
addArtifact()
functionality is now consistent across compute types
Breaking change: JDK 17 is now the default
In Databricks Runtime 16.0 and above, the default JDK version is switched from JDK 8 to JDK 17. This change is made because of the planned deprecation and end-of-support for JDK 8. This affects the following:
- Java code running on Azure Databricks compute must be compatible with Java 17.
- Scala code running in notebooks or Azure Databricks compute must be compatible with Java 17.
- Java and Scala libraries installed on compute must be compatible with Java 17.
- Versions of the Apache Hive metastore client below 2.x. Setting the Spark configuration
spark.sql.hive.metastore.version
to a version lower than 2.x will cause compatibility issues with Java 17 and connection failures to the Hive metastore. Databricks recommends upgrading Hive to a version above 2.0.0.
If you need to revert to Java 8, add the following to the Spark environment variables when you configure your Azure Databricks compute:
JNAME=zulu8-ca-amd64
If you’re using ARM instances, use the following:
JNAME=zulu8-ca-arm64
To learn more about specifying JDK versions with Azure Databricks compute, see Create a cluster that uses JDK 17.
For help with migrating your code from Java 8, see the following guides:
Breaking change: Hosted RStudio is end-of-life
With this release, Databricks-hosted RStudio Server is end-of-life and unavailable on any Azure Databricks workspace running Databricks Runtime 16.0 and above. To learn more and see a list of alternatives to RStudio, see Hosted RStudio Server deprecation.
Breaking change: Removal of support for changing byte
, short
, int
and long
types to wider types
In Databricks Runtime 15.4.3 and above, the following data type changes can no longer be applied to tables with the type widening feature enabled:
byte
,short
,int
andlong
todecimal
.byte
,short
, andint
todouble
.
This change is made to ensure consistent behavior across Delta and Iceberg tables. To learn more about type widening, see Type widening.
Correct parsing of regex patterns with negation in nested character grouping
This release includes a change to support the correct parsing of regex patterns with negation in nested character grouping. For example, [^[abc]]
will be parsed as “any character that is NOT one of ‘abc’”.
Additionally, Photon behavior was inconsistent with Spark for nested character classes. Regex patterns containing nested character classes will no longer use Photon, and instead will use Spark. A nested character class is any pattern containing square brackets within square brackets, such as [[a-c][1-3]]
.
Improve duplicate match detection in Delta Lake MERGE
In Databricks Runtime 15.4 LTS and below, MERGE
operations fail if more than one row in the source table matches the same row in the target table based on the MERGE
condition specified in the ON
clause. In Databricks Runtime 16.0 and above, MERGE
also considers conditions specified in the WHEN MATCHED
clause. See Upsert into a Delta Lake table using merge.
Cluster library installation method can no longer be overridden
The Spark configurations spark.databricks.libraries.enableSparkPyPI
, spark.databricks.libraries.enableMavenResolution
, and spark.databricks.libraries.enableCRANResolutionnow
now default to true
and cannot be overridden.
Default timeout of two hours for cluster-scoped library installs
In Databricks Runtime 16.0 and above, cluster-scoped library installation has a default timeout of two hours. Library installations that take longer than this timeout will fail, and the installation is terminated. When configuring a cluster, you can change the timeout period using the Spark configuration spark.databricks.driver.clusterLibraryInstallationTimeoutSec
.
Installing libraries from DBFS and setting the spark conf spark.databricks.driver.dbfsLibraryInstallationAllowed
are disabled
In Databricks Runtime 16.0 and above, installing libraries from DBFS is fully disabled. This change is made to improve the security of libraries in a Databricks workspace. Additionally, in Databricks Runtime 16.0 and above, you can no longer use the Spark configuration spark.databricks.driver.dbfsLibraryInstallationAllowed
.
The addArtifact()
functionality is now consistent across compute types
With this release, the archive is automatically unpacked when you use addArtifact(archive = True)
to add a dependency to shared or serverless Azure Databricks compute. This change makes the `addArtifact(archive = True) behavior on these compute types consistent with single user compute, which already supports automatically unpacking archives.
New features and improvements
- More reliable reloading of modified Python modules with improvements to
autoreload
- Avro support for recursive schema
- Expanded support for Confluent Schema Registry for Avro
- Force reclustering on tables with liquid clustering
- The Delta APIs for Python and Scala now support identity columns
- Fine-grained access control on single user compute is generally available
- Create liquid clustered tables during streaming writes
- Support for the OPTIMIZE FULL clause
- Support for WITH options specification in INSERT and table-reference
- New SQL functions
- Enable automatic schema evolution when merging data into a Delta table
More reliable reloading of modified Python modules with improvements to autoreload
In Databricks Runtime 16.0 and above, updates to the autoreload
extension improve the safety and reliability of reloading modified Python modules imported from workspace files. With these changes, autoreload
, when possible, reloads only the portion of a module that has changed instead of the entire module. Additionally, Azure Databricks now automatically suggests using the autoreload
extension if the module has changed since its last import. See Autoreload for Python modules.
Avro support for recursive schema
You can now use the recursiveFieldMaxDepth
option with the from_avro
function and the avro
data source. This option sets the maximum depth for schema recursion on the Avro data source. See Read and write streaming Avro data.
Expanded support for Confluent Schema Registry for Avro
Azure Databricks now supports Avro schema reference with the Confluent Schema Registry. See Authenticate to an external Confluent Schema Registry.
Force reclustering on tables with liquid clustering
In Databricks Runtime 16.0 and above, you can use the OPTIMIZE FULL
syntax to force the reclustering of all records in a table with liquid clustering enabled. See Force reclustering for all records.
The Delta APIs for Python and Scala now support identity columns
You can now use the Delta APIs for Python and Scala to create tables with identity columns. See Use identity columns in Delta Lake.
Fine-grained access control on single user compute is generally available
In Databricks Runtime 16.0 and above, fine-grained access control on single user compute is generally available. In workspaces enabled for serverless compute, if a query is run on supported compute such as single user compute and the query accesses any of the following objects, the compute resource passes the query to the serverless compute to run data filtering:
- Views defined over tables on which the user does not have the
SELECT
privilege. - Dynamic views.
- Tables with row filters or column masks applied.
- Materialized views and Streaming tables.
Create liquid clustered tables during streaming writes
You can now use clusterBy
to enable liquid clustering when creating new tables with Structured Streaming writes. See Enable liquid clustering.
Support for the OPTIMIZE FULL clause
Databricks Runtime 16.0 supports the OPTIMIZE FULL clause. This clause optimizes all records in a table that uses liquid clustering, including data that might have previously been clustered.
Support for WITH options specification in INSERT and table-reference
Databricks Runtime 16.0 supports an options specification for table references and table names of an INSERT
statement which can be used to control the behavior of data sources.
New SQL functions
The following SQL functions are added in Databricks Runtime 16.0:
-
This function is an error-tolerant version of url_decode. This function returns
NULL
if the input is not a valid URL-encoded string. -
If the input expression to the
zeroifnull()
function isNULL
, then the function returns 0. Otherwise, the value of the input expression is returned. -
Returns
NULL
if the input is 0 or its input if it is not 0. If the input expression to thenullifzero()
function is 0, then the function returnsNULL
. If the input expression is not 0, the value of the input expression is returned
Enable automatic schema evolution when merging data into a Delta table
This release adds support for the withSchemaEvolution()
member of the DeltaMergeBuilder
class. Use withSchemaEvolution()
to enable automatic schema evolution during MERGE
operations. For example, mergeBuilder.whenMatched(...).withSchemaEvolution().execute()}}
.
Other changes
SparkR is now deprecated
In Databricks Runtime 16.0 and above, SparkR in Databricks is deprecated in preparation for its deprecation in the upcoming Spark 4 release. See the Apache Spark Deprecate SparkR thread.
Databricks recommends using sparklyr instead.
Databricks Runtime 16.0 is not supported with PVC
Databricks Runtime 16.0 is not supported by Databricks Private Virtual Cloud (PVC). You must use Databricks Runtime 15.4 or below with all PVC releases.
Bug fixes
Auto Loader now rescues Avro record types with empty schemas
When loading an Avro file into a Delta table using Auto Loader, record
types in the file that have an empty schema are now added to the rescued data column. Because you cannot ingest empty complex data types into a Delta table, this resolves an issue with loading some Avro files. To learn more about rescued data, see What is the rescued data column?.
Fix for error writing timestamps with timezones containing a second offset.
This release fixes a bug affecting some timestamps with timezones containing a second offset. This bug causes the seconds to be omitted when writing to JSON, XML, or CSV, leading to incorrect timestamp values.
To return to the previous behavior, use the following option when writing to one of the affected formats: .option("yyyy-MM-dd'T'HH:mm:ss[.SSS][XXX]")
.
Library upgrades
- Upgraded Python libraries:
- azure-core from 1.30.2 to 1.31.0
- azure-storage-blob from 12.19.1 to 12.23.0
- azure-storage-file-datalake from 12.14.0 to 12.17.0
- black from 23.3.0 to 24.4.2
- blinker from 1.4 to 1.7.0
- boto3 from 1.34.39 to 1.34.69
- botocore from 1.34.39 to 1.34.69
- certifi from 2023.7.22 to 2024.6.2
- cffi from 1.15.1 to 1.16.0
- click from 8.0.4 to 8.1.7
- comm from 0.1.2 to 0.2.1
- contourpy from 1.0.5 to 1.2.0
- cryptography from 41.0.3 to 42.0.5
- Cython from 0.29.32 to 3.0.11
- databricks-sdk from 0.20.0 to 0.30.0
- dbus-python from 1.2.18 to 1.3.2
- filelock from 3.13.4 to 3.15.4
- fonttools from 4.25.0 to 4.51.0
- GitPython from 3.1.43 to 3.1.37
- google-api-core from 2.18.0 to 2.20.0
- google-auth from 2.31.0 to 2.35.0
- google-cloud-storage from 2.17.0 to 2.18.2
- google-crc32c from 1.5.0 to 1.6.0
- google-resumable-media from 2.7.1 to 2.7.2
- googleapis-common-protos from 1.63.2 to 1.65.0
- httplib2 from 0.20.2 to 0.20.4
- idna from 3.4 to 3.7
- ipykernel from 6.25.1 to 6.28.0
- ipython from 8.15.0 to 8.25.0
- jedi from 0.18.1 to 0.19.1
- jmespath from 0.10.0 to 1.0.1
- joblib from 1.2.0 to 1.4.2
- jupyter_client from 7.4.9 to 8.6.0
- jupyter_core from 5.3.0 to 5.7.2
- launchpadlib from 1.10.16 to 1.11.0
- lazr.restfulclient from 0.14.4 to 0.14.6
- matplotlib from 3.7.2 to 3.8.4
- mlflow-skinny from 2.11.4 to 2.15.1
- more-itertools from 8.10.0 to 10.3.0
- mypy-extensions from 0.4.3 to 1.0.0
- nest-asyncio from 1.5.6 to 1.6.0
- numpy from 1.23.5 to 1.26.4
- oauthlib from 3.2.0 to 3.2.2
- packaging from 23.2 to 24.1
- patsy from 0.5.3 to 0.5.6
- pip from 23.2.1 to 24.2
- plotly from 5.9.0 to 5.22.0
- prompt-toolkit from 3.0.36 to 3.0.43
- pyarrow from 14.0.1 to 15.0.2
- pydantic from 1.10.6 to 2.8.2
- PyGObject from 3.42.1 to 3.48.2
- PyJWT from 2.3.0 to 2.7.0
- pyodbc from 4.0.38 to 5.0.1
- python-dateutil from 2.8.2 to 2.9.0.post0
- python-lsp-jsonrpc from 1.1.1 to 1.1.2
- pytz from 2022.7 to 2024.1
- PyYAML from 6.0 to 6.0.1
- pyzmq from 23.2.0 to 25.1.2
- requests from 2.31.0 to 2.32.2
- scikit-learn from 1.3.0 to 1.4.2
- scipy from 1.11.1 to 1.13.1
- seaborn from 0.12.2 to 0.13.2
- setuptools from 68.0.0 to 74.0.0
- smmap from 5.0.1 to 5.0.0
- sqlparse from 0.5.0 to 0.5.1
- statsmodels from 0.14.0 to 0.14.2
- tornado from 6.3.2 to 6.4.1
- traitlets from 5.7.1 to 5.14.3
- typing_extensions from 4.10.0 to 4.11.0
- ujson from 5.4.0 to 5.10.0
- virtualenv from 20.24.2 to 20.26.2
- wheel from 0.38.4 to 0.43.0
- zipp from 3.11.0 to 3.17.0
- Upgraded R libraries:
- arrow from 14.0.0.2 to 16.1.0
- backports from 1.4.1 to 1.5.0
- base from 4.3.2 to 4.4.0
- bitops from 1.0-7 to 1.0-8
- boot from 1.3-28 to 1.3-30
- brio from 1.1.4 to 1.1.5
- broom from 1.0.5 to 1.0.6
- bslib from 0.6.1 to 0.8.0
- cachem from 1.0.8 to 1.1.0
- callr from 3.7.3 to 3.7.6
- cli from 3.6.2 to 3.6.3
- clock from 0.7.0 to 0.7.1
- cluster from 2.1.4 to 2.1.6
- codetools from 0.2-19 to 0.2-20
- colorspace from 2.1-0 to 2.1-1
- compiler from 4.3.2 to 4.4.0
- crayon from 1.5.2 to 1.5.3
- curl from 5.2.0 to 5.2.1
- data.table from 1.15.0 to 1.15.4
- datasets from 4.3.2 to 4.4.0
- DBI from 1.2.1 to 1.2.3
- dbplyr from 2.4.0 to 2.5.0
- digest from 0.6.34 to 0.6.36
- downlit from 0.4.3 to 0.4.4
- evaluate from 0.23 to 0.24.0
- farver from 2.1.1 to 2.1.2
- fastmap from 1.1.1 to 1.2.0
- foreign from 0.8-85 to 0.8-86
- fs from 1.6.3 to 1.6.4
- future from 1.33.1 to 1.34.0
- future.apply from 1.11.1 to 1.11.2
- gert from 2.0.1 to 2.1.0
- ggplot2 from 3.4.4 to 3.5.1
- gh from 1.4.0 to 1.4.1
- globals from 0.16.2 to 0.16.3
- graphics from 4.3.2 to 4.4.0
- grDevices from 4.3.2 to 4.4.0
- grid from 4.3.2 to 4.4.0
- gt from 0.10.1 to 0.11.0
- gtable from 0.3.4 to 0.3.5
- hardhat from 1.3.1 to 1.4.0
- highr from 0.10 to 0.11
- htmltools from 0.5.7 to 0.5.8.1
- httpuv from 1.6.14 to 1.6.15
- httr2 from 1.0.0 to 1.0.2
- ipred from 0.9-14 to 0.9-15
- KernSmooth from 2.23-21 to 2.23-22
- knitr from 1.45 to 1.48
- lattice from 0.21-8 to 0.22-5
- lava from 1.7.3 to 1.8.0
- markdown from 1.12 to 1.13
- MASS from 7.3-60 to 7.3-60.0.1
- Matrix from 1.5-4.1 to 1.6-5
- methods from 4.3.2 to 4.4.0
- mgcv from 1.8-42 to 1.9-1
- mlflow from 2.10.0 to 2.14.1
- munsell from 0.5.0 to 0.5.1
- nlme from 3.1-163 to 3.1-165
- openssl from 2.1.1 to 2.2.0
- parallel from 4.3.2 to 4.4.0
- parallelly from 1.36.0 to 1.38.0
- pkgbuild from 1.4.3 to 1.4.4
- pkgdown from 2.0.7 to 2.1.0
- pkgload from 1.3.4 to 1.4.0
- processx from 3.8.3 to 3.8.4
- prodlim from 2023.08.28 to 2024.06.25
- promises from 1.2.1 to 1.3.0
- ps from 1.7.6 to 1.7.7
- ragg from 1.2.7 to 1.3.2
- Rcpp from 1.0.12 to 1.0.13
- RcppEigen from 0.3.3.9.4 to 0.3.4.0.0
- reactR from 0.5.0 to 0.6.0
- recipes from 1.0.9 to 1.1.0
- remotes from 2.4.2.1 to 2.5.0
- reprex from 2.1.0 to 2.1.1
- rlang from 1.1.3 to 1.1.4
- rmarkdown from 2.25 to 2.27
- roxygen2 from 7.3.1 to 7.3.2
- rpart from 4.1.21 to 4.1.23
- RSQLite from 2.3.5 to 2.3.7
- rstudioapi from 0.15.0 to 0.16.0
- rvest from 1.0.3 to 1.0.4
- sass from 0.4.8 to 0.4.9
- shape from 1.4.6 to 1.4.6.1
- shiny from 1.8.0 to 1.9.1
- sparklyr from 1.8.4 to 1.8.6
- spatial from 7.3-15 to 7.3-17
- splines from 4.3.2 to 4.4.0
- stats from 4.3.2 to 4.4.0
- stats4 from 4.3.2 to 4.4.0
- stringi from 1.8.3 to 1.8.4
- survival from 3.5-5 to 3.6-4
- swagger from 3.33.1 to 5.17.14.1
- systemfonts from 1.0.5 to 1.1.0
- tcltk from 4.3.2 to 4.4.0
- testthat from 3.2.1 to 3.2.1.1
- textshaping from 0.3.7 to 0.4.0
- tidyselect from 1.2.0 to 1.2.1
- tinytex from 0.49 to 0.52
- tools from 4.3.2 to 4.4.0
- usethis from 2.2.2 to 3.0.0
- utils from 4.3.2 to 4.4.0
- uuid from 1.2-0 to 1.2-1
- V8 from 4.4.1 to 4.4.2
- withr from 3.0.0 to 3.0.1
- xfun from 0.41 to 0.46
- xopen from 1.0.0 to 1.0.1
- yaml from 2.3.8 to 2.3.10
- Upgraded Java libraries:
- com.amazonaws.aws-java-sdk-autoscaling from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-cloudformation from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-cloudfront from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-cloudhsm from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-cloudsearch from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-cloudtrail from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-cloudwatch from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-cloudwatchmetrics from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-codedeploy from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-cognitoidentity from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-cognitosync from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-config from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-core from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-datapipeline from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-directconnect from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-directory from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-dynamodb from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-ec2 from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-ecs from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-efs from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-elasticache from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-elasticbeanstalk from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-elasticloadbalancing from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-elastictranscoder from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-emr from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-glacier from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-glue from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-iam from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-importexport from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-kinesis from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-kms from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-lambda from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-logs from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-machinelearning from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-opsworks from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-rds from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-redshift from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-route53 from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-s3 from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-ses from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-simpledb from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-simpleworkflow from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-sns from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-sqs from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-ssm from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-storagegateway from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-sts from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-support from 1.12.610 to 1.12.638
- com.amazonaws.aws-java-sdk-workspaces from 1.12.610 to 1.12.638
- com.amazonaws.jmespath-java from 1.12.610 to 1.12.638
- com.google.protobuf.protobuf-java from 2.6.1 to 3.25.1
- io.airlift.aircompressor from 0.25 to 0.27
- io.delta.delta-sharing-client_2.12 from 1.1.3 to 1.2.0
- io.netty.netty-all from 4.1.96.Final to 4.1.108.Final
- io.netty.netty-buffer from 4.1.96.Final to 4.1.108.Final
- io.netty.netty-codec from 4.1.96.Final to 4.1.108.Final
- io.netty.netty-codec-http from 4.1.96.Final to 4.1.108.Final
- io.netty.netty-codec-http2 from 4.1.96.Final to 4.1.108.Final
- io.netty.netty-codec-socks from 4.1.96.Final to 4.1.108.Final
- io.netty.netty-common from 4.1.96.Final to 4.1.108.Final
- io.netty.netty-handler from 4.1.96.Final to 4.1.108.Final
- io.netty.netty-handler-proxy from 4.1.96.Final to 4.1.108.Final
- io.netty.netty-resolver from 4.1.96.Final to 4.1.108.Final
- io.netty.netty-transport from 4.1.96.Final to 4.1.108.Final
- io.netty.netty-transport-classes-epoll from 4.1.96.Final to 4.1.108.Final
- io.netty.netty-transport-classes-kqueue from 4.1.96.Final to 4.1.108.Final
- io.netty.netty-transport-native-epoll from 4.1.96.Final-linux-x86_64 to 4.1.108.Final-linux-x86_64
- io.netty.netty-transport-native-kqueue from 4.1.96.Final-osx-x86_64 to 4.1.108.Final-osx-x86_64
- io.netty.netty-transport-native-unix-common from 4.1.96.Final to 4.1.108.Final
- org.apache.ivy.ivy from 2.5.1 to 2.5.2
- org.apache.zookeeper.zookeeper from 3.6.3 to 3.9.2
- org.apache.zookeeper.zookeeper-jute from 3.6.3 to 3.9.2
- org.rocksdb.rocksdbjni from 8.11.4 to 9.2.1
- org.scalactic.scalactic_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest-compatible from 3.2.15 to 3.2.16
- org.scalatest.scalatest-core_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest-diagrams_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest-featurespec_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest-flatspec_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest-freespec_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest-funspec_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest-funsuite_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest-matchers-core_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest-mustmatchers_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest-propspec_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest-refspec_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest-shouldmatchers_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest-wordspec_2.12 from 3.2.15 to 3.2.16
- org.scalatest.scalatest_2.12 from 3.2.15 to 3.2.16
Apache Spark
Databricks Runtime 16.0 includes Apache Spark 3.5.0. This release includes all Spark fixes and improvements included in Databricks Runtime 15.4 LTS, as well as the following additional bug fixes and improvements made to Spark:
- [SPARK-49093] [DBRRM-1371] Revert “[SC-172958][SQL] GROUP BY with MapType nes…
- [SPARK-49898] [DBRRM-1282][SC-178410] Fix documentation and default for event log task metrics accumulator logging flag from SPARK-42204
- [SPARK-49743] [ES-1260022][BEHAVE-157][SC-177475][SQL] OptimizeCsvJsonExpr should not change schema fields when pruning GetArrayStructFields
- [SPARK-49816] [SC-177896][SQL] Should only update out-going-ref-count for referenced outer CTE relation
- [SPARK-48939] [SC-177022][SC-172766][AVRO] Support reading Avro with recursive schema reference
- [SPARK-49688] [SC-177468][ES-1242349][CONNECT] Fix a data race between interrupt and execute plan
- [SPARK-49771] [SC-177466][PYTHON] Improve Pandas Scalar Iter UDF error when output rows exceed input rows
- [SPARK-48866] [SC-170772][SQL] Fix hints of valid charset in the error message of INVALID_PARAMETER_VALUE.CHARSET
- [SPARK-48195] [FIXFORWARD][SC-177267][CORE] Save and reuse RDD/Broadcast created by SparkPlan
- [SPARK-49585] [CONNECT] Replace executions map in SessionHolder with operationID set
- [SPARK-49211] [SC-174257][SQL] V2 Catalog can also support built-in data sources
- [SPARK-49684] Minimise the lifetime of the session restoration lock
- [SPARK-48059] [SPARK-48145][SPARK-48134][SPARK-48182][SPARK-48209][SPARK-48291] Structured log framework on the java side
- [SPARK-48857] [SC-170661][SQL] Restrict charsets in CSVOptions
- [SPARK-49152] [SC-173690][SQL] V2SessionCatalog should use V2Command
- [SPARK-42846] [SC-176588][SQL] Remove error condition _LEGACY_ERROR_TEMP_2011
- [SPARK-48195] [SC-177267][CORE] Save and reuse RDD/Broadcast created by SparkPlan
- [SPARK-49630] [SC-177379][SS] Add flatten option to process collection types with state data source reader
- [SPARK-49699] [SC-177154][SS] Disable PruneFilters for streaming workloads
- [SPARK-48781] [SC-175282][SQL] Add Catalog APIs for loading stored procedures
- [SPARK-49667] [SC-177068][SQL] Disallow CS_AI collators with expressions that use StringSearch
- [SPARK-49737] [SC-177207][SQL] Disable bucketing on collated columns in complex types
- [SPARK-48712] [SC-169794][SQL] Perf Improvement for encode with empty values or UTF-8 charset
- [SPARK-49038] [SC-173933][SQL] SQLMetric should report the raw value in the accumulator update event
- [SPARK-48541] [SC-169066][CORE] Add a new exit code for executors killed by TaskReaper
- [SPARK-48774] [SC-170138][SQL] Use SparkSession in SQLImplicits
- [SPARK-49719] [SC-177139][SQL] Make
UUID
andSHUFFLE
accept integerseed
- [SPARK-49713] [SC-177135][PYTHON][CONNECT] Make function
count_min_sketch
accept number arguments - [SPARK-47601] [SC-162499][GRAPHX] Graphx: Migrate logs with variables to structured logging framework
- [SPARK-49738] [SC-177219][SQL] Endswith bug fix
- [SPARK-48623] [SC-170822][CORE] Structured logging migrations [Part 3]
- [SPARK-49677] [SC-177148][SS] Ensure that changelog files are written on commit and forceSnapshot flag is also reset
- [SPARK-49684] [SC-177040][CONNECT] Remove global locks from session and execution managers
- [SPARK-48302] [SC-168814][PYTHON] Preserve nulls in map columns in PyArrow Tables
- [SPARK-48601] [SC-169025][SQL] Give a more user friendly error message when setting a null value for JDBC Option
- [SPARK-48635] [SC-169263][SQL] Assign classes to join type errors and as-of join error
- [SPARK-49673] [SC-177032][CONNECT] Increase CONNECT_GRPC_ARROW_MAX_BATCH_SIZE to 0.7 * CONNECT_GRPC_MAX_MESSAGE_SIZE
- [SPARK-49693] [SC-177071][PYTHON][CONNECT] Refine the string representation of
timedelta
- [SPARK-49687] [SC-176901][SQL] Delay sorting in
validateAndMaybeEvolveStateSchema
- [SPARK-49718] [SC-177112][PS] Switch
Scatter
plot to sampled data - [SPARK-48472] [SC-169044][SQL] Enable reflect expressions with collated strings
- [SPARK-48484] [SC-167484][SQL] Fix: V2Write use the same TaskAttemptId for different task attempts
- [SPARK-48341] [SC-166560][CONNECT] Allow plugins to use QueryTest in their tests
- [SPARK-42252] [SC-168723][CORE] Add
spark.shuffle.localDisk.file.output.buffer
and deprecatespark.shuffle.unsafe.file.output.buffer
- [SPARK-48314] [SC-166565][SS] Don’t double cache files for FileStreamSource using Trigger.AvailableNow
- [SPARK-49567] [SC-176241][PYTHON] Use
classic
instead ofvanilla
from PySpark code base - [SPARK-48374] [SC-167596][PYTHON] Support additional PyArrow Table column types
- [SPARK-48300] [SC-166481][SQL] Codegen Support for
from_xml
- [SPARK-49412] [SC-177059][PS] Compute all box plot metrics in single job
- [SPARK-49692] [SC-177031][PYTHON][CONNECT] Refine the string representation of literal date and datetime
- [SPARK-49392] [ES-1130351][SC-176705][SQL] Catch errors when failing to write to external data source
- [SPARK-48306] [SC-166241][SQL] Improve UDT in error message
- [SPARK-44924] [SC-166379][SS] Add config for FileStreamSource cached files
- [SPARK-48176] [SC-165644][SQL] Adjust name of FIELD_ALREADY_EXISTS error condition
- [SPARK-49691] [SC-176988][PYTHON][CONNECT] Function
substring
should accept column names - [SPARK-49502] [SC-176077][CORE] Avoid NPE in SparkEnv.get.shuffleManager.unregisterShuffle
- [SPARK-49244] [SC-176703][SQL] Further exception improvements for parser/interpreter
- [SPARK-48355] [SC-176684][SQL] Support for CASE statement
- [SPARK-49355] [SC-175121][SQL]
levenshtein
should check whether thecollation
values of all parameter types are the same - [SPARK-49640] [SC-176953][PS] Apply reservoir sampling in
SampledPlotBase
- [SPARK-49678] [SC-176857][CORE] Support
spark.test.master
inSparkSubmitArguments
- [SPARK-49680] [SC-176856][PYTHON] Limit
Sphinx
build parallelism to 4 by default - [SPARK-49396] Revert “[SC-176030][SQL] Modify nullability check for CaseWhen expression”
- [SPARK-48419] [SC-167443][SQL] Foldable propagation replace foldable column shoul…
- [SPARK-49556] [SC-176757][SQL] Add SQL pipe syntax for the SELECT operator
- [SPARK-49438] [SC-175237][SQL] Fix the pretty name of the
FromAvro
&ToAvro
expression - [SPARK-49659] [SC-1229924][SQL] Add a nice user-facing error for scalar subqueries inside VALUES clause
- [SPARK-49646] [SC-176778][SQL] fix subquery decorrelation for union/set operations when parentOuterReferences has references not covered in collectedChildOuterReferences
- [SPARK-49354] [SC-175034][SQL]
split_part
should check whether thecollation
values of all parameter types are the same - [SPARK-49478] [SC-175914][CONNECT] Handle null metrics in ConnectProgressExecutionListener
- [SPARK-48358] [SC-176374][SQL] Support for REPEAT statement
- [SPARK-49183] [SC-173680][SQL] V2SessionCatalog.createTable should respect PROP_IS_MANAGED_LOCATION
- [SPARK-49611] [SC-176791][SQL] Introduce TVF
collations()
& remove theSHOW COLLATIONS
command - [SPARK-49261] [SC-176589][SQL] Don’t replace literals in aggregate expressions with group-by expressions
- [SPARK-49099] [SC-173229][SQL] CatalogManager.setCurrentNamespace should respect custom session catalog
- [SPARK-49594] [SC-176569][SS] Adding check on whether columnFamilies were added or removed to write StateSchemaV3 file
- [SPARK-49578] [SC-176385][SQL] Remove the ANSI config suggestion in CAST_INVALID_INPUT and CAST_OVERFLOW
- [SPARK-48882] [SC-174256][SS] Assign names to streaming output mode related error classes
- [SPARK-49155] [SC-176506][SQL][SS] Use more appropriate parameter type to construct
GenericArrayData
- [SPARK-49519] [SC-176388][SQL] Merge options of table and relation when constructing FileScanBuilder
- [SPARK-49591] [SC-176587][SQL] Add Logical Type column to variant readme
- [SPARK-49596] [SC-176423][SQL] Improve performance of
FormatString
- [SPARK-49525] [SC-176044][SS][CONNECT] Minor log improvement to Server Side Streaming Query ListenerBus Listener
- [SPARK-49583] [SC-176272][SQL] Define the error sub-condition
SECONDS_FRACTION
for invalid seconds fraction pattern - [SPARK-49536] [SC-176242] Handle error in python streaming data source record prefetching
- [SPARK-49443] [SC-176273][SQL][PYTHON] Implement to_variant_object expression and make schema_of_variant expressions print OBJECT for for Variant Objects
- [SPARK-49544] [SASP-3990][SC-176557][CONNECT] Replace coarse-locking in SparkConnectExecutionManager with ConcurrentMap
- [SPARK-49548] [SASP-3990][SC-176556][CONNECT] Replace coarse-locking in SparkConnectSessionManager with ConcurrentMap
- [SPARK-49551] [SC-176218][SS] Improve RocksDB log for replayChangelog
- [SPARK-49595] [SC-176396][CONNECT][SQL] Fix
DataFrame.unpivot/melt
in Spark Connect Scala Client - [SPARK-49006] [SC-176162] Implement purging for OperatorStateMetadataV2 and StateSchemaV3 files
- [SPARK-49600] [SC-176426][PYTHON] Remove
Python 3.6 and older
-related logic fromtry_simplify_traceback
- [SPARK-49303] [SC-176013][SS] Implement TTL for ValueState in transformWithStateInPandas API
- [SPARK-49191] [SC-176243][SS] Add support for reading transformWithState map state variables with state data source reader
- [SPARK-49593] [SC-176371][SS] Throw RocksDB exception to the caller on DB close if an error is seen
- [SPARK-49334] [SC-174803][SQL]
str_to_map
should check whether thecollation
values of all parameter types are the same - [SPARK-42204] [SC-176126][CORE] Add option to disable redundant logging of TaskMetrics internal accumulators in event logs
- [SPARK-49575] [SC-176256][SS] Add logging for lock release only if acquiredThreadInfo is not null
- [SPARK-49539] [SC-176250][SS] Update internal col families start identifier to a different one
- [SPARK-49205] [SC-173853][SQL] KeyGroupedPartitioning should inherit HashPartitioningLike
- [SPARK-49396] [SC-176030][SQL] Modify nullability check for CaseWhen expression
- [SPARK-49476] [SC-175700][SQL] Fix nullability of base64 function
- [SPARK-47262] [SC-174829][SQL] Assign names to error conditions for parquet conversions
- [SPARK-47247] [SC-158349][SQL] Use smaller target size when coalescing partitions with exploding joins
- [SPARK-49501] [SC-176166][SQL] Fix double-escaping of table location
- [SPARK-49083] [SC-173214][CONNECT] Allow from_xml and from_json to natively work with json schemas
- [SPARK-49043] [SC-174673][SQL] Fix interpreted codepath group by on map containing collated strings
- [SPARK-48986] [SC-172265][CONNECT][SQL] Add ColumnNode Intermediate Representation
- [SPARK-49326] [SC-176016][SS] Classify Error class for Foreach sink user function error
- [SPARK-48348] [SC-175950][SPARK-48376][SQL] Introduce
LEAVE
andITERATE
statements - [SPARK-49523] [SC-175949][CONNECT] Increase maximum wait time for connect server to come up for testing
- [SPARK-49000] [BEHAVE-105][ES-1194747][SQL] Fix “select count(distinct 1) from t” where t is empty table by expanding RewriteDistinctAggregates - DBR version 16.x
- [SPARK-49311] [SC-175038][SQL] Make it possible for large ‘interval second’ values to be cast to decimal
- [SPARK-49200] [SC-173699][SQL] Fix null type non-codegen ordering exception
- [SPARK-49467] [SC-176051][SS] Add support for state data source reader and list state
- [SPARK-47307] [SC-170891][SQL] Add a config to optionally chunk base64 strings
- [SPARK-49391] [SC-176032][PS] Box plot select outliers by distance from fences
- [SPARK-49445] [SC-175845][UI] Support show tooltip in the progress bar of UI
- [SPARK-49451] [SC-175702] Allow duplicate keys in parse_json.
- [SPARK-49275] [SC-175701][SQL] Fix return type nullness of the xpath expression
- [SPARK-49021] [SC-175578][SS] Add support for reading transformWithState value state variables with state data source reader
- [SPARK-49474] [BEHAVE-143][SC-169253][SC-175933][SS] Classify Error class for FlatMapGroupsWithState user function error
- [SPARK-49408] [SC-175932][SQL] Use IndexedSeq in ProjectingInternalRow
- [SPARK-49509] [SC-175853][CORE] Use
Platform.allocateDirectBuffer
instead ofByteBuffer.allocateDirect
- [SPARK-49382] [SC-175013][PS] Make frame box plot properly render the fliers/outliers
- [SPARK-49002] [SC-172846][SQL] Consistently handle invalid locations in WAREHOUSE/SCHEMA/TABLE/PARTITION/DIRECTORY
- [SPARK-49480] [SC-175699][CORE] Fix NullPointerException from
SparkThrowableHelper.isInternalError
- [SPARK-49477] [SC-175828][PYTHON] Improve pandas udf invalid return type error message
- [SPARK-48693] [SC-169492][SQL] Simplify and unify toString of Invoke and StaticInvoke
- [SPARK-49441] [SC-175716][ML]
StringIndexer
sort arrays in executors - [SPARK-49347] [SC-175004][R] Deprecate SparkR
- [SPARK-49357] [SC-175227][CONNECT][PYTHON] Vertically truncate deeply nested protobuf message
- [SPARK-41982] [SC-120604][SQL] Partitions of type string should not be treated as numeric types
- [SPARK-48776] [SC-170452][BEHAVE-72] Fix timestamp formatting for json, xml and csv
- [SPARK-49223] [SC-174800][ML] Simplify the StringIndexer.countByValue with builtin functions
- [SPARK-49016] Revert “[SC-174663][SQL] Restore the behavior thatqueries from raw CSV files are disallowed when only include corrupt record column and assign name to
_LEGACY_ERROR_TEMP_1285
” - [SPARK-49041] [SC-172392][PYTHON][CONNECT] Raise proper error for
dropDuplicates
when wrongsubset
is given - [SPARK-49050] [SC-175235] Enabling deleteIfExists operator in TWS with Virtual Column Families
- [SPARK-49216] [SC-173919][CORE]Fix to not log message context with explicitly LogEntry constructed when Structured Logging conf is off
- [SPARK-49252] [SC-175596][CORE] Make
TaskSetExcludeList
andHeathTracker
independent - [SPARK-49352] [SC-174971][SQL] Avoid redundant array transform for identical expression
- [SPARK-42307] [SC-173863][SQL] Assign name for error _LEGACY_ERROR_TEMP_2232
- [SPARK-49197] [SC-173732][CORE] Redact
Spark Command
output inlauncher
module - [SPARK-48913] [SC-173934][SQL] Implement IndentingXMLStreamWriter
- [SPARK-49306] [SC-175363][PYTHON][SQL] Create SQL function aliases for ‘zeroifnull’ and ‘nullifzero’
- [SPARK-48344] [SQL] SQL Scripting execution (including Spark Connect)
- [SPARK-49402] [SC-175122][PYTHON] Fix Binder integration in PySpark documentation
- [SPARK-49017] [SC-174664][SQL] Insert statement fails when multiple parameters are being used
- [SPARK-49318] [SC-174733][SQL] Preempt low priority error on LCA until end of check analysis to improve error experience
- [SPARK-49016] [SC-174663][SQL] Restore the behavior that queries from raw CSV files are disallowed when only include corrupt record column and assign name to
_LEGACY_ERROR_TEMP_1285
- [SPARK-49387] [SC-175124][PYTHON] Fix type hint for
accuracy
inpercentile_approx
andapprox_percentile
- [SPARK-49131] [SC-174666][SS] TransformWithState should properly set implicit grouping keys even with lazy iterators
- [SPARK-49301] [SC-174795][SS] Chunk arrow data passed to Python worker
- [SPARK-49039] [SC-174651][UI] Reset checkbox when executor metrics are loaded in the Stages tab
- [SPARK-48428] [SC-169806][SQL]: Fix IllegalStateException in NestedColumnAliasing
- [SPARK-49353] [SC-174830][SQL] Update docs related to
UTF-32
encoding/decoding - [SPARK-48613] [SC-170966][SQL] SPJ: Support auto-shuffle one side + less join keys than partition keys
- [SPARK-47473] [SC-160450][BEHAVE-127][SQL] Fix correctness issue of converting postgres INFINITY timestamps
- [SPARK-49142] [SC-173658][CONNECT][PYTHON] Follow up to revert proto to string performance cost
- [SPARK-49300] [SC-175008][CORE] Fix Hadoop delegation token leak when tokenRenewalInterval is not set.
- [SPARK-49367] [SC-175012][PS] Parallelize the KDE computation for multiple columns (plotly backend)
- [SPARK-49365] [SC-175011][PS] Simplify the bucket aggregation in hist plot
- [SPARK-49372] [SC-175003][SS] Ensure that latestSnapshot is set to none on close to avoid subsequent use
- [SPARK-49341] [SC-174785] Remove
connector/docker
in favor ofApache Spark Operator
- [SPARK-49344] [SC-174894][PS] Support
json_normalize
for Pandas API on Spark - [SPARK-49306] [SC-174794][SQL] Create new SQL functions ‘zeroifnull’ and ‘nullifzero’
- [SPARK-48796] [SC-174668][SS] Load Column Family Id from RocksDBCheckpointMetadata for VCF when restarting
- [SPARK-49342] [SC-174899][SQL] Make TO_AVRO SQL function ‘jsonFormatSchema’ argument optional
- [SPARK-48628] [SC-174695][CORE] Add task peak on/off heap memory metrics
- [SPARK-47407] [SC-159379][BEHAVE-126][SQL] Support java.sql.Types.NULL map to NullType
- [SPARK-48628] [SC-173407][CORE] Add task peak on/off heap memory metrics
- [SPARK-49166] [SC-173987][SQL] Support OFFSET in correlated subquery
- [SPARK-49269] [SC-174676][SQL] Eagerly evaluate VALUES() list in AstBuilder
- [SPARK-49281] [SC-174782][SQL] Optimze parquet binary getBytes with getBytesUnsafe to avoid copy cost
- [SPARK-49113] [SC-174734] Dont assert on translation bugs - silently swallow the exception
- [SPARK-49098] [SC-173253][SQL] Add write options for INSERT
- [SPARK-48638] [SC-174694][FOLLOW][CONNECT] Fix documentation for ExecutionInfo
- [SPARK-49250] [ES-1222826][SQL] Improve error message for nested UnresolvedWindowExpression in CheckAnalysis
- [SPARK-48755] [SC-174258][SS][PYTHON] transformWithState pyspark base implementation and ValueState support
- [SPARK-48966] [SC-174329][SQL] Improve error message with invalid unresolved column reference in UDTF call
- [SPARK-46590] [SC-154115][SQL] Fix coalesce failed with unexpected partition indeces
- [SPARK-49235] [SC-174159][SQL] Refactor ResolveInlineTables rule so it doesn’t traverse the whole tree
- [SPARK-49060] [SC-173107][CONNECT] Clean up Mima rules for SQL-Connect binary compatibility checks
- [SPARK-48762] [SC-172525][SQL] Introduce clusterBy DataFrameWriter API for Python
- [SPARK-49207] [SC-173852][SQL] Fix one-to-many case mapping in SplitPart and StringSplitSQL
- [SPARK-49204] [SC-173850][SQL] Fix surrogate pair handling in StringInstr and StringLocate
- [SPARK-36680] [SC-170640][SQL] Supports Dynamic Table Options for Spark SQL
- [SPARK-49204] [SC-173849][SQL] Fix surrogate pair handling in SubstringIndex
- [SPARK-49204] [SC-173848][SQL] Fix surrogate pair handling in StringTrim
- [SPARK-48967] [SC-173993]Fix forward SparkConfigOwnershipSuite test for OPTIMIZE_INSERT_INTO_VALUES_PARSER
- [SPARK-49204] [SC-173851][SQL] Fix surrogate pair handling in StringReplace
- [SPARK-48967] [SC-173993][SQL][16.x] Improve performance and memory footprint of “INSERT INTO … VALUES” Statements
- [SPARK-49099] Revert “[SC-173229][SQL] CatalogManager.setCurrent…
- [SPARK-48347] [SC-173812][SQL] Support for WHILE statement
- [SPARK-49128] [SC-173344][CORE] Support custom History Server UI title
- [SPARK-49146] [SC-173825][SS] Move assertion errors related to watermark missing in append mode streaming queries to error framework
- [SPARK-45787] [SC-172197][SQL] Support Catalog.listColumns for clustering columns
- [SPARK-49099] [SC-173229][SQL] CatalogManager.setCurrentNamespace should respect custom session catalog
- [SPARK-49138] [SC-173483][SQL] Fix CollationTypeCasts of several expressions
- [SPARK-49163] [SC-173666][SQL] Attempt to create table based on broken parquet partition data results should return user-facing error
- [SPARK-49201] [SC-173793][PS][PYTHON][CONNECT] Reimplement
hist
plot with Spark SQL - [SPARK-49188] [SC-173682][SQL] Internal error on concat_ws called on array of arrays of string
- [SPARK-49137] [SC-173677][SQL] When the Boolean condition in the
if statement
is invalid, an exception should be thrown - [SPARK-49193] [SC-173672][SQL] Improve the performance of RowSetUtils.toColumnBasedSet
- [SPARK-49078] [SC-173078][SQL] Support show columns syntax in v2 table
- [SPARK-49141] [SC-173388][SQL] Mark variant as hive incompatible data type
- [SPARK-49059] [Cherry-Pick][15.x][SC-172528][CONNECT] Move
SessionHolder.forTesting(...)
to the test package - [SPARK-49111] [SC-173661][SQL] Move withProjectAndFilter to the companion object of DataSourceV2Strategy
- [SPARK-49185] [SC-173688][PS][PYTHON][CONNECT] Reimplement
kde
plot with Spark SQL - [SPARK-49178] [SC-173673][SQL] Optimize performance of
Row#getSeq
to match the performance when using Spark 3.5 with Scala 2.12 - [SPARK-49093] [SC-172958][SQL] GROUP BY with MapType nested inside complex type
- [SPARK-49142] [SC-173469][CONNECT][PYTHON] Lower Spark Connect client log level to debug
- [SPARK-48761] [SC-172048][SQL] Introduce clusterBy DataFrameWriter API for Scala
- [SPARK-48346] [SC-173083][SQL] Support for IF ELSE statements in SQL scripts
- [SPARK-48338] [SC-173112][SQL] Improve exceptions thrown from parser/interpreter
- [SPARK-48658] [SC-169474][SQL] Encode/Decode functions report coding errors instead of mojibake for unmappable characters
- [SPARK-49071] [SC-172954][SQL] Remove ArraySortLike trait
- [SPARK-49107] Revert “Revert “[SC-173103][SQL]
ROUTINE_ALREADY_EXISTS
supports RoutineType”” - [SPARK-49070] [SC-172907][SS][SQL] TransformWithStateExec.initialState is rewritten incorrectly to produce invalid query plan
- [SPARK-49114] [SC-173217] Sub-categorize cannot load state store errors
- [SPARK-49107] Revert “[SC-173103][SQL]
ROUTINE_ALREADY_EXISTS
supports RoutineType” - [SPARK-49048] [SC-173223][SS] Add support for reading relevant operator metadata at given batch id
- [SPARK-49094] [SC-173049][SQL] Fix ignoreCorruptFiles non-functioning for hive orc impl with mergeSchema off
- [SPARK-49108] [SC-173102][EXAMPLE] Add
submit_pi.sh
REST API example - [SPARK-49107] [SC-173103][SQL]
ROUTINE_ALREADY_EXISTS
supports RoutineType - [SPARK-48997] [SC-172484][SS] Implement individual unloads for maintenance thread pool thread failures
- [SPARK-49063] [SC-173094][SQL] Fix Between with ScalarSubqueries
- [SPARK-45891] [SC-172305][SQL][PYTHON][VARIANT] Add support for interval types in the Variant Spec
- [SPARK-49074] [BEHAVE-110][SC-172815][SQL] Fix variant with
df.cache()
- [SPARK-49003] [SC-172613][SQL] Fix interpreted code path hashing to be collation aware
- [SPARK-48740] [SC-172430][SQL] Catch missing window specification error early
- [SPARK-48999] [SC-172245][SS] Divide PythonStreamingDataSourceSimpleSuite
- [SPARK-49031] [SC-172602] Implement validation for the TransformWithStateExec operator using OperatorStateMetadataV2
- [SPARK-49053] [SC-172494][PYTHON][ML] Make model save/load helper functions accept spark session
- [SPARK-49032] [Backport][15.x][SS] Add schema path in metadata table entry, verify expected version and add operator metadata related test for operator metadata format v2
- [SPARK-49034] [SC-172306][CORE] Support server-side
sparkProperties
replacement in REST Submission API - [SPARK-48931] [SC-171895][SS] Reduce Cloud Store List API cost for state store maintenance task
- [SPARK-48849] [SC-172068][SS]Create OperatorStateMetadataV2 for the TransformWithStateExec operator
- [SPARK-49013] [SC-172322] Change key in collationsMap for Map and Array types in scala
- [SPARK-48414] [SC-171884][PYTHON] Fix breaking change in python’s
fromJson
- [SPARK-48910] [SC-171001][SQL] Use HashSet/HashMap to avoid linear searches in PreprocessTableCreation
- [SPARK-49007] [SC-172204][CORE] Improve
MasterPage
to support custom title - [SPARK-49009] [SC-172263][SQL][PYTHON] Make Column APIs and functions accept Enums
- [SPARK-49033] [SC-172303][CORE] Support server-side
environmentVariables
replacement in REST Submission API - [SPARK-48363] [SC-166470][SQL] Cleanup some redundant codes in
from_xml
- [SPARK-46743] [SC-170867][SQL][BEHAVE-84] Count bug after ScalarSubqery is folded if it has an empty relation
- [SPARK-49040] [SC-172351][SQL] Fix doc
sql-ref-syntax-aux-exec-imm.md
- [SPARK-48998] [SC-172212][ML] Meta algorithms save/load model with SparkSession
- [SPARK-48959] [SC-171708][SQL] Make
NoSuchNamespaceException
extendNoSuchDatabaseException
to restore the exception handling - [SPARK-48996] [SC-172130][SQL][PYTHON] Allow bare literals for and and or of Column
- [SPARK-48990] [SC-171936] Follow up for #101759 - test fix
- [SPARK-48338] [SC-171912][SQL] Check variable declarations
- [SPARK-48990] [SC-171936][SQL] Unified variable related SQL syntax keywords
- [SPARK-48988] [SC-171915][ML] Make
DefaultParamsReader/Writer
handle metadata with spark session - [SPARK-48974] [SC-171978][SQL][SS][ML][MLLIB] Use
SparkSession.implicits
instead ofSQLContext.implicits
- [SPARK-48760] [SC-170870][SQL] Fix CatalogV2Util.applyClusterByChanges
- [SPARK-48928] [SC-171956] Log Warning for Calling .unpersist() on Locally Checkpointed RDDs
- [SPARK-48760] [SC-170139][SQL] Introduce ALTER TABLE … CLUSTER BY SQL syntax to change clustering columns
- [SPARK-48844] Revert “[SC-170669][SQL] USE INVALID_EMPTY_LOCATION instead of UNSUPPORTED_DATASOURCE_FOR_DIRECT_QUERY when path is empty”
- [SPARK-48833] [SC-171914][SQL][VARIANT] Support variant in
InMemoryTableScan
- [SPARK-48975] [SC-171894][PROTOBUF] Remove unnecessary
ScalaReflectionLock
definition fromprotobuf
- [SPARK-48970] [SC-171800][PYTHON][ML] Avoid using SparkSession.getActiveSession in spark ML reader/writer
- [SPARK-48844] [SC-170669][SQL] USE INVALID_EMPTY_LOCATION instead of UNSUPPORTED_DATASOURCE_FOR_DIRECT_QUERY when path is empty
- [SPARK-48714] [SC-170136] Fix failing df.mergeInto tests in PySpark and UC
- [SPARK-48957] [SC-171797][SS] Return sub-classified error class on state store load for hdfs and rocksdb provider
- [SPARK-48891] [Backport][15x][SC-171677][SS] Refactor StateSchemaCompatibilityChecker to unify all state schema formats
- [SPARK-48972] [SC-171795][PYTHON] Unify the literal string handling in functions
- [SPARK-48388] [SC-171337][SQL] Fix SET statement behavior for SQL Scripts
- [SPARK-48743] [SC-170552][SQL][SS] MergingSessionIterator should better handle when getStruct returns null
- [SPARK-48623] [15.x][SC-171322][CORE] Migrate FileAppender logs to structured logging
- [SPARK-36680] [DBRRM-1123] Revert “[SC-170640][SQL] Supports Dynamic Table Options for Spark SQL”
- [SPARK-48841] [SC-170868][BEHAVE-83][SQL] Include
collationName
tosql()
ofCollate
- [SPARK-48941] [SC-171703][PYTHON][ML] Replace RDD read / write API invocation with Dataframe read / write API
- [SPARK-48938] [SC-171577][PYTHON] Improve error messages when registering Python UDTFs
- [SPARK-48350] [SC-171040][SQL] Introduction of Custom Exceptions for Sql Scripting
- [SPARK-48907] [SC-171158][SQL] Fix the value
explicitTypes
inCOLLATION_MISMATCH.EXPLICIT
- [SPARK-48945] [SC-171658][PYTHON] Simplify regex functions with
lit
- [SPARK-48944] [SC-171576][CONNECT] Unify the JSON-format schema handling in Connect Server
- [SPARK-48836] [SC-171569] Integrate SQL schema with state schema/metadata
- [SPARK-48946] [SC-171504][SQL] NPE in redact method when session is null
- [SPARK-48921] [SC-171412][SQL] ScalaUDF encoders in subquery should be resolved for MergeInto
- [SPARK-45155] [SC-171048][CONNECT] Add API Docs for Spark Connect JVM/Scala Client
- [SPARK-48900] [SC-171319] Add
reason
field forcancelJobGroup
andcancelJobsWithTag
- [SPARK-48865] [SC-171154][SQL] Add try_url_decode function
- [SPARK-48851] [SC-170767][SQL] Change the value of
SCHEMA_NOT_FOUND
fromnamespace
tocatalog.namespace
- [SPARK-48510] [SC-170893][2/2] Support UDAF
toColumn
API in Spark Connect - [SPARK-45190] [SC-171055][SPARK-48897][PYTHON][CONNECT] Make
from_xml
support StructType schema - [SPARK-48930] [SC-171304][CORE] Redact
awsAccessKeyId
by includingaccesskey
pattern - [SPARK-48909] [SC-171080][ML][MLLIB] Uses SparkSession over SparkContext when writing metadata
- [SPARK-48883] [SC-171133][ML][R] Replace RDD read / write API invocation with Dataframe read / write API
- [SPARK-48924] [SC-171313][PS] Add a pandas-like
make_interval
helper function - [SPARK-48884] [SC-171051][PYTHON] Remove unused helper function
PythonSQLUtils.makeInterval
- [SPARK-48817] [SC-170636][SQL] Eagerly execute union multi commands together
- [SPARK-48896] [SC-171079][ML][MLLIB] Avoid repartition when writing out the metadata
- [SPARK-48892] [SC-171127][ML] Avoid per-row param read in
Tokenizer
- [SPARK-48927] [SC-171227][CORE] Show the number of cached RDDs in
StoragePage
- [SPARK-48886] [15.x][Backport][SC-171039][SS] Add version info to changelog v2 to allow for easier evolution
- [SPARK-48903] [SC-171136][SS] Set the RocksDB last snapshot version correctly on remote load
- [SPARK-48742] [SC-170538][SS] Virtual Column Family for RocksDB
- [SPARK-48726] [15.x][SC-170753][SS] Create the StateSchemaV3 file format, and write this out for the TransformWithStateExec operator
- [SPARK-48794] [SC-170882][CONNECT][15.x] df.mergeInto support for Spark Connect (Scala and Python)
- [SPARK-48714] [SC-170136][PYTHON] Implement
DataFrame.mergeInto
in PySpark - [SPARK-48772] [SC-170642][SS][SQL] State Data Source Change Feed Reader Mode
- [SPARK-48666] [SC-170887][SQL] Do not push down filter if it contains PythonUDFs
- [SPARK-48845] [SC-170889][SQL] GenericUDF catch exceptions from children
- [SPARK-48880] [SC-170974][CORE] Avoid throw NullPointerException if driver plugin fails to initialize
- [SPARK-48888] [Backport][15x][SC-170973][SS] Remove snapshot creation based on changelog ops size
- [SPARK-48871] [SC-170876] Fix INVALID_NON_DETERMINISTIC_EXPRESSIONS validation in…
- [SPARK-48883] [SC-170894][ML][R] Replace RDD read / write API invocation with Dataframe read / write API
- [SPARK-36680] [SC-170640][SQL] Supports Dynamic Table Options for Spark SQL
- [SPARK-48804] [SC-170558][SQL] Add classIsLoadable & OutputCommitter.isAssignableFrom check for output committer class configrations
- [SPARK-46738] [SC-170791][PYTHON] Reenable a group of doctests
- [SPARK-48858] [SC-170756][PYTHON] Remove deprecated
setDaemon
method call ofThread
inlog_communication.py
- [SPARK-48639] [SC-169801][CONNECT][PYTHON] Add Origin to RelationCommon
- [SPARK-48863] [SC-170770][ES-1133940][SQL] Fix ClassCastException when parsing JSON with “spark.sql.json.enablePartialResults” enabled
- [SPARK-48343] [SC-170450][SQL] Introduction of SQL Scripting interpreter
- [SPARK-48529] [SC-170755][SQL] Introduction of Labels in SQL Scripting
- [SPARK-45292] Revert “[SC-151609][SQL][HIVE] Remove Guava from shared classes from IsolatedClientLoader”
- [SPARK-48037] [SC-165330][CORE][3.5] Fix SortShuffleWriter lacks shuffle write related metrics resulting in potentially inaccurate data
- [SPARK-48720] [SC-170551][SQL] Align the command
ALTER TABLE ... UNSET TBLPROPERTIES ...
in v1 and v2 - [SPARK-48485] [SC-167825][CONNECT][SS] Support interruptTag and interruptAll in streaming queries
- [SPARK-45292] [SC-151609][SQL][HIVE] Remove Guava from shared classes from IsolatedClientLoader
- [SPARK-48668] [SC-169815][SQL] Support ALTER NAMESPACE … UNSET PROPERTIES in v2
- [SPARK-47914] [SC-165313][SQL] Do not display the splits parameter in Range
- [SPARK-48807] [SC-170643][SQL] Binary Support for CSV datasource
- [SPARK-48220] [SC-167592][PYTHON][15.X] Allow passing PyArrow Table to createDataFrame()
- [SPARK-48545] [SC-169543][SQL] Create to_avro and from_avro SQL functions to match DataFrame equivalents
- [SPARK-47577] [SC-168875][SPARK-47579] Correct misleading usage of log key TASK_ID
Databricks ODBC/JDBC driver support
Databricks supports ODBC/JDBC drivers released in the past 2 years. Please download the recently released drivers and upgrade (download ODBC, download JDBC).
System environment
- Operating System: Ubuntu 24.04.1 LTS
- Java: Zulu17.50+19-CA
- Scala: 2.12.18
- Python: 3.12.3
- R: 4.4.0
- Delta Lake: 3.2.1
Installed Python libraries
Library | Version | Library | Version | Library | Version |
---|---|---|---|---|---|
annotated-types | 0.7.0 | asttokens | 2.0.5 | astunparse | 1.6.3 |
autocommand | 2.2.2 | azure-core | 1.31.0 | azure-storage-blob | 12.23.0 |
azure-storage-file-datalake | 12.17.0 | backports.tarfile | 1.2.0 | black | 24.4.2 |
blinker | 1.7.0 | boto3 | 1.34.69 | botocore | 1.34.69 |
cachetools | 5.3.3 | certifi | 2024.6.2 | cffi | 1.16.0 |
chardet | 4.0.0 | charset-normalizer | 2.0.4 | click | 8.1.7 |
cloudpickle | 2.2.1 | comm | 0.2.1 | contourpy | 1.2.0 |
cryptography | 42.0.5 | cycler | 0.11.0 | Cython | 3.0.11 |
databricks-sdk | 0.30.0 | dbus-python | 1.3.2 | debugpy | 1.6.7 |
decorator | 5.1.1 | Deprecated | 1.2.14 | distlib | 0.3.8 |
docstring-to-markdown | 0.11 | entrypoints | 0.4 | executing | 0.8.3 |
facets-overview | 1.1.1 | filelock | 3.15.4 | fonttools | 4.51.0 |
gitdb | 4.0.11 | GitPython | 3.1.37 | google-api-core | 2.20.0 |
google-auth | 2.35.0 | google-cloud-core | 2.4.1 | google-cloud-storage | 2.18.2 |
google-crc32c | 1.6.0 | google-resumable-media | 2.7.2 | googleapis-common-protos | 1.65.0 |
grpcio | 1.60.0 | grpcio-status | 1.60.0 | httplib2 | 0.20.4 |
idna | 3.7 | importlib-metadata | 6.0.0 | importlib_resources | 6.4.0 |
inflect | 7.3.1 | ipyflow-core | 0.0.198 | ipykernel | 6.28.0 |
ipython | 8.25.0 | ipython-genutils | 0.2.0 | ipywidgets | 7.7.2 |
isodate | 0.6.1 | jaraco.context | 5.3.0 | jaraco.functools | 4.0.1 |
jaraco.text | 3.12.1 | jedi | 0.19.1 | jmespath | 1.0.1 |
joblib | 1.4.2 | jupyter_client | 8.6.0 | jupyter_core | 5.7.2 |
kiwisolver | 1.4.4 | launchpadlib | 1.11.0 | lazr.restfulclient | 0.14.6 |
lazr.uri | 1.0.6 | matplotlib | 3.8.4 | matplotlib-inline | 0.1.6 |
mccabe | 0.7.0 | mlflow-skinny | 2.15.1 | more-itertools | 10.3.0 |
mypy | 1.10.0 | mypy-extensions | 1.0.0 | nest-asyncio | 1.6.0 |
nodeenv | 1.9.1 | numpy | 1.26.4 | oauthlib | 3.2.2 |
opentelemetry-api | 1.27.0 | opentelemetry-sdk | 1.27.0 | opentelemetry-semantic-conventions | 0.48b0 |
packaging | 24.1 | pandas | 1.5.3 | parso | 0.8.3 |
pathspec | 0.10.3 | patsy | 0.5.6 | pexpect | 4.8.0 |
pillow | 10.3.0 | pip | 24.2 | platformdirs | 3.10.0 |
plotly | 5.22.0 | pluggy | 1.0.0 | prompt-toolkit | 3.0.43 |
proto-plus | 1.24.0 | protobuf | 4.24.1 | psutil | 5.9.0 |
psycopg2 | 2.9.3 | ptyprocess | 0.7.0 | pure-eval | 0.2.2 |
pyarrow | 15.0.2 | pyasn1 | 0.4.8 | pyasn1-modules | 0.2.8 |
pyccolo | 0.0.52 | pycparser | 2.21 | pydantic | 2.8.2 |
pydantic_core | 2.20.1 | pyflakes | 3.2.0 | Pygments | 2.15.1 |
PyGObject | 3.48.2 | PyJWT | 2.7.0 | pyodbc | 5.0.1 |
pyparsing | 3.0.9 | pyright | 1.1.294 | python-dateutil | 2.9.0.post0 |
python-lsp-jsonrpc | 1.1.2 | python-lsp-server | 1.10.0 | pytoolconfig | 1.2.6 |
pytz | 2024.1 | PyYAML | 6.0.1 | pyzmq | 25.1.2 |
requests | 2.32.2 | rope | 1.12.0 | rsa | 4.9 |
s3transfer | 0.10.2 | scikit-learn | 1.4.2 | scipy | 1.13.1 |
seaborn | 0.13.2 | setuptools | 74.0.0 | six | 1.16.0 |
smmap | 5.0.0 | sqlparse | 0.5.1 | ssh-import-id | 5.11 |
stack-data | 0.2.0 | statsmodels | 0.14.2 | tenacity | 8.2.2 |
threadpoolctl | 2.2.0 | tokenize-rt | 4.2.1 | tomli | 2.0.1 |
tornado | 6.4.1 | traitlets | 5.14.3 | typeguard | 4.3.0 |
types-protobuf | 3.20.3 | types-psutil | 5.9.0 | types-pytz | 2023.3.1.1 |
types-PyYAML | 6.0.0 | types-requests | 2.31.0.0 | types-setuptools | 68.0.0.0 |
types-six | 1.16.0 | types-urllib3 | 1.26.25.14 | typing_extensions | 4.11.0 |
ujson | 5.10.0 | unattended-upgrades | 0.1 | urllib3 | 1.26.16 |
virtualenv | 20.26.2 | wadllib | 1.3.6 | wcwidth | 0.2.5 |
whatthepatch | 1.0.2 | wheel | 0.43.0 | wrapt | 1.14.1 |
yapf | 0.33.0 | zipp | 3.17.0 |
Installed R libraries
R libraries are installed from the Posit Package Manager CRAN snapshot.
Library | Version | Library | Version | Library | Version |
---|---|---|---|---|---|
arrow | 16.1.0 | askpass | 1.2.0 | assertthat | 0.2.1 |
backports | 1.5.0 | base | 4.4.0 | base64enc | 0.1-3 |
bigD | 0.2.0 | bit | 4.0.5 | bit64 | 4.0.5 |
bitops | 1.0-8 | blob | 1.2.4 | boot | 1.3-30 |
brew | 1.0-10 | brio | 1.1.5 | broom | 1.0.6 |
bslib | 0.8.0 | cachem | 1.1.0 | callr | 3.7.6 |
caret | 6.0-94 | cellranger | 1.1.0 | chron | 2.3-61 |
class | 7.3-22 | cli | 3.6.3 | clipr | 0.8.0 |
clock | 0.7.1 | cluster | 2.1.6 | codetools | 0.2-20 |
colorspace | 2.1-1 | commonmark | 1.9.1 | compiler | 4.4.0 |
config | 0.3.2 | conflicted | 1.2.0 | cpp11 | 0.4.7 |
crayon | 1.5.3 | credentials | 2.0.1 | curl | 5.2.1 |
data.table | 1.15.4 | datasets | 4.4.0 | DBI | 1.2.3 |
dbplyr | 2.5.0 | desc | 1.4.3 | devtools | 2.4.5 |
diagram | 1.6.5 | diffobj | 0.3.5 | digest | 0.6.36 |
downlit | 0.4.4 | dplyr | 1.1.4 | dtplyr | 1.3.1 |
e1071 | 1.7-14 | ellipsis | 0.3.2 | evaluate | 0.24.0 |
fansi | 1.0.6 | farver | 2.1.2 | fastmap | 1.2.0 |
fontawesome | 0.5.2 | forcats | 1.0.0 | foreach | 1.5.2 |
foreign | 0.8-86 | forge | 0.2.0 | fs | 1.6.4 |
future | 1.34.0 | future.apply | 1.11.2 | gargle | 1.5.2 |
generics | 0.1.3 | gert | 2.1.0 | ggplot2 | 3.5.1 |
gh | 1.4.1 | git2r | 0.33.0 | gitcreds | 0.1.2 |
glmnet | 4.1-8 | globals | 0.16.3 | glue | 1.7.0 |
googledrive | 2.1.1 | googlesheets4 | 1.1.1 | gower | 1.0.1 |
graphics | 4.4.0 | grDevices | 4.4.0 | grid | 4.4.0 |
gridExtra | 2.3 | gsubfn | 0.7 | gt | 0.11.0 |
gtable | 0.3.5 | hardhat | 1.4.0 | haven | 2.5.4 |
highr | 0.11 | hms | 1.1.3 | htmltools | 0.5.8.1 |
htmlwidgets | 1.6.4 | httpuv | 1.6.15 | httr | 1.4.7 |
httr2 | 1.0.2 | ids | 1.0.1 | ini | 0.3.1 |
ipred | 0.9-15 | isoband | 0.2.7 | iterators | 1.0.14 |
jquerylib | 0.1.4 | jsonlite | 1.8.8 | juicyjuice | 0.1.0 |
KernSmooth | 2.23-22 | knitr | 1.48 | labeling | 0.4.3 |
later | 1.3.2 | lattice | 0.22-5 | lava | 1.8.0 |
lifecycle | 1.0.4 | listenv | 0.9.1 | lubridate | 1.9.3 |
magrittr | 2.0.3 | markdown | 1.13 | MASS | 7.3-60.0.1 |
Matrix | 1.6-5 | memoise | 2.0.1 | methods | 4.4.0 |
mgcv | 1.9-1 | mime | 0.12 | miniUI | 0.1.1.1 |
mlflow | 2.14.1 | ModelMetrics | 1.2.2.2 | modelr | 0.1.11 |
munsell | 0.5.1 | nlme | 3.1-165 | nnet | 7.3-19 |
numDeriv | 2016.8-1.1 | openssl | 2.2.0 | parallel | 4.4.0 |
parallelly | 1.38.0 | pillar | 1.9.0 | pkgbuild | 1.4.4 |
pkgconfig | 2.0.3 | pkgdown | 2.1.0 | pkgload | 1.4.0 |
plogr | 0.2.0 | plyr | 1.8.9 | praise | 1.0.0 |
prettyunits | 1.2.0 | pROC | 1.18.5 | processx | 3.8.4 |
prodlim | 2024.06.25 | profvis | 0.3.8 | progress | 1.2.3 |
progressr | 0.14.0 | promises | 1.3.0 | proto | 1.0.0 |
proxy | 0.4-27 | ps | 1.7.7 | purrr | 1.0.2 |
R6 | 2.5.1 | ragg | 1.3.2 | randomForest | 4.7-1.1 |
rappdirs | 0.3.3 | rcmdcheck | 1.4.0 | RColorBrewer | 1.1-3 |
Rcpp | 1.0.13 | RcppEigen | 0.3.4.0.0 | reactable | 0.4.4 |
reactR | 0.6.0 | readr | 2.1.5 | readxl | 1.4.3 |
recipes | 1.1.0 | rematch | 2.0.0 | rematch2 | 2.1.2 |
remotes | 2.5.0 | reprex | 2.1.1 | reshape2 | 1.4.4 |
rlang | 1.1.4 | rmarkdown | 2.27 | RODBC | 1.3-23 |
roxygen2 | 7.3.2 | rpart | 4.1.23 | rprojroot | 2.0.4 |
Rserve | 1.8-13 | RSQLite | 2.3.7 | rstudioapi | 0.16.0 |
rversions | 2.1.2 | rvest | 1.0.4 | sass | 0.4.9 |
scales | 1.3.0 | selectr | 0.4-2 | sessioninfo | 1.2.2 |
shape | 1.4.6.1 | shiny | 1.9.1 | sourcetools | 0.1.7-1 |
sparklyr | 1.8.6 | spatial | 7.3-17 | splines | 4.4.0 |
sqldf | 0.4-11 | SQUAREM | 2021.1 | stats | 4.4.0 |
stats4 | 4.4.0 | stringi | 1.8.4 | stringr | 1.5.1 |
survival | 3.6-4 | swagger | 5.17.14.1 | sys | 3.4.2 |
systemfonts | 1.1.0 | tcltk | 4.4.0 | testthat | 3.2.1.1 |
textshaping | 0.4.0 | tibble | 3.2.1 | tidyr | 1.3.1 |
tidyselect | 1.2.1 | tidyverse | 2.0.0 | timechange | 0.3.0 |
timeDate | 4032.109 | tinytex | 0.52 | tools | 4.4.0 |
tzdb | 0.4.0 | urlchecker | 1.0.1 | usethis | 3.0.0 |
utf8 | 1.2.4 | utils | 4.4.0 | uuid | 1.2-1 |
V8 | 4.4.2 | vctrs | 0.6.5 | viridisLite | 0.4.2 |
vroom | 1.6.5 | waldo | 0.5.2 | whisker | 0.4.1 |
withr | 3.0.1 | xfun | 0.46 | xml2 | 1.3.6 |
xopen | 1.0.1 | xtable | 1.8-4 | yaml | 2.3.10 |
zeallot | 0.1.0 | zip | 2.3.1 |
Installed Java and Scala libraries (Scala 2.12 cluster version)
Group ID | Artifact ID | Version |
---|---|---|
antlr | antlr | 2.7.7 |
com.amazonaws | amazon-kinesis-client | 1.12.0 |
com.amazonaws | aws-java-sdk-autoscaling | 1.12.638 |
com.amazonaws | aws-java-sdk-cloudformation | 1.12.638 |
com.amazonaws | aws-java-sdk-cloudfront | 1.12.638 |
com.amazonaws | aws-java-sdk-cloudhsm | 1.12.638 |
com.amazonaws | aws-java-sdk-cloudsearch | 1.12.638 |
com.amazonaws | aws-java-sdk-cloudtrail | 1.12.638 |
com.amazonaws | aws-java-sdk-cloudwatch | 1.12.638 |
com.amazonaws | aws-java-sdk-cloudwatchmetrics | 1.12.638 |
com.amazonaws | aws-java-sdk-codedeploy | 1.12.638 |
com.amazonaws | aws-java-sdk-cognitoidentity | 1.12.638 |
com.amazonaws | aws-java-sdk-cognitosync | 1.12.638 |
com.amazonaws | aws-java-sdk-config | 1.12.638 |
com.amazonaws | aws-java-sdk-core | 1.12.638 |
com.amazonaws | aws-java-sdk-datapipeline | 1.12.638 |
com.amazonaws | aws-java-sdk-directconnect | 1.12.638 |
com.amazonaws | aws-java-sdk-directory | 1.12.638 |
com.amazonaws | aws-java-sdk-dynamodb | 1.12.638 |
com.amazonaws | aws-java-sdk-ec2 | 1.12.638 |
com.amazonaws | aws-java-sdk-ecs | 1.12.638 |
com.amazonaws | aws-java-sdk-efs | 1.12.638 |
com.amazonaws | aws-java-sdk-elasticache | 1.12.638 |
com.amazonaws | aws-java-sdk-elasticbeanstalk | 1.12.638 |
com.amazonaws | aws-java-sdk-elasticloadbalancing | 1.12.638 |
com.amazonaws | aws-java-sdk-elastictranscoder | 1.12.638 |
com.amazonaws | aws-java-sdk-emr | 1.12.638 |
com.amazonaws | aws-java-sdk-glacier | 1.12.638 |
com.amazonaws | aws-java-sdk-glue | 1.12.638 |
com.amazonaws | aws-java-sdk-iam | 1.12.638 |
com.amazonaws | aws-java-sdk-importexport | 1.12.638 |
com.amazonaws | aws-java-sdk-kinesis | 1.12.638 |
com.amazonaws | aws-java-sdk-kms | 1.12.638 |
com.amazonaws | aws-java-sdk-lambda | 1.12.638 |
com.amazonaws | aws-java-sdk-logs | 1.12.638 |
com.amazonaws | aws-java-sdk-machinelearning | 1.12.638 |
com.amazonaws | aws-java-sdk-opsworks | 1.12.638 |
com.amazonaws | aws-java-sdk-rds | 1.12.638 |
com.amazonaws | aws-java-sdk-redshift | 1.12.638 |
com.amazonaws | aws-java-sdk-route53 | 1.12.638 |
com.amazonaws | aws-java-sdk-s3 | 1.12.638 |
com.amazonaws | aws-java-sdk-ses | 1.12.638 |
com.amazonaws | aws-java-sdk-simpledb | 1.12.638 |
com.amazonaws | aws-java-sdk-simpleworkflow | 1.12.638 |
com.amazonaws | aws-java-sdk-sns | 1.12.638 |
com.amazonaws | aws-java-sdk-sqs | 1.12.638 |
com.amazonaws | aws-java-sdk-ssm | 1.12.638 |
com.amazonaws | aws-java-sdk-storagegateway | 1.12.638 |
com.amazonaws | aws-java-sdk-sts | 1.12.638 |
com.amazonaws | aws-java-sdk-support | 1.12.638 |
com.amazonaws | aws-java-sdk-swf-libraries | 1.11.22 |
com.amazonaws | aws-java-sdk-workspaces | 1.12.638 |
com.amazonaws | jmespath-java | 1.12.638 |
com.clearspring.analytics | stream | 2.9.6 |
com.databricks | Rserve | 1.8-3 |
com.databricks | databricks-sdk-java | 0.27.0 |
com.databricks | jets3t | 0.7.1-0 |
com.databricks.scalapb | scalapb-runtime_2.12 | 0.4.15-10 |
com.esotericsoftware | kryo-shaded | 4.0.2 |
com.esotericsoftware | minlog | 1.3.0 |
com.fasterxml | classmate | 1.3.4 |
com.fasterxml.jackson.core | jackson-annotations | 2.15.2 |
com.fasterxml.jackson.core | jackson-core | 2.15.2 |
com.fasterxml.jackson.core | jackson-databind | 2.15.2 |
com.fasterxml.jackson.dataformat | jackson-dataformat-cbor | 2.15.2 |
com.fasterxml.jackson.dataformat | jackson-dataformat-yaml | 2.15.2 |
com.fasterxml.jackson.datatype | jackson-datatype-joda | 2.15.2 |
com.fasterxml.jackson.datatype | jackson-datatype-jsr310 | 2.16.0 |
com.fasterxml.jackson.module | jackson-module-paranamer | 2.15.2 |
com.fasterxml.jackson.module | jackson-module-scala_2.12 | 2.15.2 |
com.github.ben-manes.caffeine | caffeine | 2.9.3 |
com.github.fommil | jniloader | 1.1 |
com.github.fommil.netlib | native_ref-java | 1.1 |
com.github.fommil.netlib | native_ref-java | 1.1-natives |
com.github.fommil.netlib | native_system-java | 1.1 |
com.github.fommil.netlib | native_system-java | 1.1-natives |
com.github.fommil.netlib | netlib-native_ref-linux-x86_64 | 1.1-natives |
com.github.fommil.netlib | netlib-native_system-linux-x86_64 | 1.1-natives |
com.github.luben | zstd-jni | 1.5.5-4 |
com.github.wendykierp | JTransforms | 3.1 |
com.google.code.findbugs | jsr305 | 3.0.0 |
com.google.code.gson | gson | 2.10.1 |
com.google.crypto.tink | tink | 1.9.0 |
com.google.errorprone | error_prone_annotations | 2.10.0 |
com.google.flatbuffers | flatbuffers-java | 23.5.26 |
com.google.guava | guava | 15.0 |
com.google.protobuf | protobuf-java | 3.25.1 |
com.helger | profiler | 1.1.1 |
com.ibm.icu | icu4j | 75.1 |
com.jcraft | jsch | 0.1.55 |
com.jolbox | bonecp | 0.8.0.RELEASE |
com.lihaoyi | sourcecode_2.12 | 0.1.9 |
com.microsoft.azure | azure-data-lake-store-sdk | 2.3.9 |
com.microsoft.sqlserver | mssql-jdbc | 11.2.2.jre8 |
com.ning | compress-lzf | 1.1.2 |
com.sun.mail | javax.mail | 1.5.2 |
com.sun.xml.bind | jaxb-core | 2.2.11 |
com.sun.xml.bind | jaxb-impl | 2.2.11 |
com.tdunning | json | 1.8 |
com.thoughtworks.paranamer | paranamer | 2.8 |
com.trueaccord.lenses | lenses_2.12 | 0.4.12 |
com.twitter | chill-java | 0.10.0 |
com.twitter | chill_2.12 | 0.10.0 |
com.twitter | util-app_2.12 | 7.1.0 |
com.twitter | util-core_2.12 | 7.1.0 |
com.twitter | util-function_2.12 | 7.1.0 |
com.twitter | util-jvm_2.12 | 7.1.0 |
com.twitter | util-lint_2.12 | 7.1.0 |
com.twitter | util-registry_2.12 | 7.1.0 |
com.twitter | util-stats_2.12 | 7.1.0 |
com.typesafe | config | 1.4.3 |
com.typesafe.scala-logging | scala-logging_2.12 | 3.7.2 |
com.uber | h3 | 3.7.3 |
com.univocity | univocity-parsers | 2.9.1 |
com.zaxxer | HikariCP | 4.0.3 |
commons-cli | commons-cli | 1.5.0 |
commons-codec | commons-codec | 1.16.0 |
commons-collections | commons-collections | 3.2.2 |
commons-dbcp | commons-dbcp | 1.4 |
commons-fileupload | commons-fileupload | 1.5 |
commons-httpclient | commons-httpclient | 3.1 |
commons-io | commons-io | 2.13.0 |
commons-lang | commons-lang | 2.6 |
commons-logging | commons-logging | 1.1.3 |
commons-pool | commons-pool | 1.5.4 |
dev.ludovic.netlib | arpack | 3.0.3 |
dev.ludovic.netlib | blas | 3.0.3 |
dev.ludovic.netlib | lapack | 3.0.3 |
info.ganglia.gmetric4j | gmetric4j | 1.0.10 |
io.airlift | aircompressor | 0.27 |
io.delta | delta-sharing-client_2.12 | 1.2.0 |
io.dropwizard.metrics | metrics-annotation | 4.2.19 |
io.dropwizard.metrics | metrics-core | 4.2.19 |
io.dropwizard.metrics | metrics-graphite | 4.2.19 |
io.dropwizard.metrics | metrics-healthchecks | 4.2.19 |
io.dropwizard.metrics | metrics-jetty9 | 4.2.19 |
io.dropwizard.metrics | metrics-jmx | 4.2.19 |
io.dropwizard.metrics | metrics-json | 4.2.19 |
io.dropwizard.metrics | metrics-jvm | 4.2.19 |
io.dropwizard.metrics | metrics-servlets | 4.2.19 |
io.netty | netty-all | 4.1.108.Final |
io.netty | netty-buffer | 4.1.108.Final |
io.netty | netty-codec | 4.1.108.Final |
io.netty | netty-codec-http | 4.1.108.Final |
io.netty | netty-codec-http2 | 4.1.108.Final |
io.netty | netty-codec-socks | 4.1.108.Final |
io.netty | netty-common | 4.1.108.Final |
io.netty | netty-handler | 4.1.108.Final |
io.netty | netty-handler-proxy | 4.1.108.Final |
io.netty | netty-resolver | 4.1.108.Final |
io.netty | netty-tcnative-boringssl-static | 2.0.61.Final |
io.netty | netty-tcnative-boringssl-static | 2.0.61.Final-linux-aarch_64 |
io.netty | netty-tcnative-boringssl-static | 2.0.61.Final-linux-x86_64 |
io.netty | netty-tcnative-boringssl-static | 2.0.61.Final-osx-aarch_64 |
io.netty | netty-tcnative-boringssl-static | 2.0.61.Final-osx-x86_64 |
io.netty | netty-tcnative-boringssl-static | 2.0.61.Final-windows-x86_64 |
io.netty | netty-tcnative-classes | 2.0.61.Final |
io.netty | netty-transport | 4.1.108.Final |
io.netty | netty-transport-classes-epoll | 4.1.108.Final |
io.netty | netty-transport-classes-kqueue | 4.1.108.Final |
io.netty | netty-transport-native-epoll | 4.1.108.Final |
io.netty | netty-transport-native-epoll | 4.1.108.Final-linux-aarch_64 |
io.netty | netty-transport-native-epoll | 4.1.108.Final-linux-riscv64 |
io.netty | netty-transport-native-epoll | 4.1.108.Final-linux-x86_64 |
io.netty | netty-transport-native-kqueue | 4.1.108.Final-osx-aarch_64 |
io.netty | netty-transport-native-kqueue | 4.1.108.Final-osx-x86_64 |
io.netty | netty-transport-native-unix-common | 4.1.108.Final |
io.prometheus | simpleclient | 0.7.0 |
io.prometheus | simpleclient_common | 0.7.0 |
io.prometheus | simpleclient_dropwizard | 0.7.0 |
io.prometheus | simpleclient_pushgateway | 0.7.0 |
io.prometheus | simpleclient_servlet | 0.7.0 |
io.prometheus.jmx | collector | 0.12.0 |
jakarta.annotation | jakarta.annotation-api | 1.3.5 |
jakarta.servlet | jakarta.servlet-api | 4.0.3 |
jakarta.validation | jakarta.validation-api | 2.0.2 |
jakarta.ws.rs | jakarta.ws.rs-api | 2.1.6 |
javax.activation | activation | 1.1.1 |
javax.el | javax.el-api | 2.2.4 |
javax.jdo | jdo-api | 3.0.1 |
javax.transaction | jta | 1.1 |
javax.transaction | transaction-api | 1.1 |
javax.xml.bind | jaxb-api | 2.2.11 |
javolution | javolution | 5.5.1 |
jline | jline | 2.14.6 |
joda-time | joda-time | 2.12.1 |
net.java.dev.jna | jna | 5.8.0 |
net.razorvine | pickle | 1.3 |
net.sf.jpam | jpam | 1.1 |
net.sf.opencsv | opencsv | 2.3 |
net.sf.supercsv | super-csv | 2.2.0 |
net.snowflake | snowflake-ingest-sdk | 0.9.6 |
net.sourceforge.f2j | arpack_combined_all | 0.1 |
org.acplt.remotetea | remotetea-oncrpc | 1.1.2 |
org.antlr | ST4 | 4.0.4 |
org.antlr | antlr-runtime | 3.5.2 |
org.antlr | antlr4-runtime | 4.9.3 |
org.antlr | stringtemplate | 3.2.1 |
org.apache.ant | ant | 1.10.11 |
org.apache.ant | ant-jsch | 1.10.11 |
org.apache.ant | ant-launcher | 1.10.11 |
org.apache.arrow | arrow-format | 15.0.0 |
org.apache.arrow | arrow-memory-core | 15.0.0 |
org.apache.arrow | arrow-memory-netty | 15.0.0 |
org.apache.arrow | arrow-vector | 15.0.0 |
org.apache.avro | avro | 1.11.3 |
org.apache.avro | avro-ipc | 1.11.3 |
org.apache.avro | avro-mapred | 1.11.3 |
org.apache.commons | commons-collections4 | 4.4 |
org.apache.commons | commons-compress | 1.23.0 |
org.apache.commons | commons-crypto | 1.1.0 |
org.apache.commons | commons-lang3 | 3.12.0 |
org.apache.commons | commons-math3 | 3.6.1 |
org.apache.commons | commons-text | 1.10.0 |
org.apache.curator | curator-client | 2.13.0 |
org.apache.curator | curator-framework | 2.13.0 |
org.apache.curator | curator-recipes | 2.13.0 |
org.apache.datasketches | datasketches-java | 3.1.0 |
org.apache.datasketches | datasketches-memory | 2.0.0 |
org.apache.derby | derby | 10.14.2.0 |
org.apache.hadoop | hadoop-client-runtime | 3.3.6 |
org.apache.hive | hive-beeline | 2.3.9 |
org.apache.hive | hive-cli | 2.3.9 |
org.apache.hive | hive-jdbc | 2.3.9 |
org.apache.hive | hive-llap-client | 2.3.9 |
org.apache.hive | hive-llap-common | 2.3.9 |
org.apache.hive | hive-serde | 2.3.9 |
org.apache.hive | hive-shims | 2.3.9 |
org.apache.hive | hive-storage-api | 2.8.1 |
org.apache.hive.shims | hive-shims-0.23 | 2.3.9 |
org.apache.hive.shims | hive-shims-common | 2.3.9 |
org.apache.hive.shims | hive-shims-scheduler | 2.3.9 |
org.apache.httpcomponents | httpclient | 4.5.14 |
org.apache.httpcomponents | httpcore | 4.4.16 |
org.apache.ivy | ivy | 2.5.2 |
org.apache.logging.log4j | log4j-1.2-api | 2.22.1 |
org.apache.logging.log4j | log4j-api | 2.22.1 |
org.apache.logging.log4j | log4j-core | 2.22.1 |
org.apache.logging.log4j | log4j-layout-template-json | 2.22.1 |
org.apache.logging.log4j | log4j-slf4j2-impl | 2.22.1 |
org.apache.orc | orc-core | 1.9.2-shaded-protobuf |
org.apache.orc | orc-mapreduce | 1.9.2-shaded-protobuf |
org.apache.orc | orc-shims | 1.9.2 |
org.apache.thrift | libfb303 | 0.9.3 |
org.apache.thrift | libthrift | 0.12.0 |
org.apache.ws.xmlschema | xmlschema-core | 2.3.0 |
org.apache.xbean | xbean-asm9-shaded | 4.23 |
org.apache.yetus | audience-annotations | 0.13.0 |
org.apache.zookeeper | zookeeper | 3.9.2 |
org.apache.zookeeper | zookeeper-jute | 3.9.2 |
org.checkerframework | checker-qual | 3.31.0 |
org.codehaus.jackson | jackson-core-asl | 1.9.13 |
org.codehaus.jackson | jackson-mapper-asl | 1.9.13 |
org.codehaus.janino | commons-compiler | 3.0.16 |
org.codehaus.janino | janino | 3.0.16 |
org.datanucleus | datanucleus-api-jdo | 4.2.4 |
org.datanucleus | datanucleus-core | 4.1.17 |
org.datanucleus | datanucleus-rdbms | 4.1.19 |
org.datanucleus | javax.jdo | 3.2.0-m3 |
org.eclipse.collections | eclipse-collections | 11.1.0 |
org.eclipse.collections | eclipse-collections-api | 11.1.0 |
org.eclipse.jetty | jetty-client | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-continuation | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-http | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-io | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-jndi | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-plus | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-proxy | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-security | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-server | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-servlet | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-servlets | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-util | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-util-ajax | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-webapp | 9.4.52.v20230823 |
org.eclipse.jetty | jetty-xml | 9.4.52.v20230823 |
org.eclipse.jetty.websocket | websocket-api | 9.4.52.v20230823 |
org.eclipse.jetty.websocket | websocket-client | 9.4.52.v20230823 |
org.eclipse.jetty.websocket | websocket-common | 9.4.52.v20230823 |
org.eclipse.jetty.websocket | websocket-server | 9.4.52.v20230823 |
org.eclipse.jetty.websocket | websocket-servlet | 9.4.52.v20230823 |
org.fusesource.leveldbjni | leveldbjni-all | 1.8 |
org.glassfish.hk2 | hk2-api | 2.6.1 |
org.glassfish.hk2 | hk2-locator | 2.6.1 |
org.glassfish.hk2 | hk2-utils | 2.6.1 |
org.glassfish.hk2 | osgi-resource-locator | 1.0.3 |
org.glassfish.hk2.external | aopalliance-repackaged | 2.6.1 |
org.glassfish.hk2.external | jakarta.inject | 2.6.1 |
org.glassfish.jersey.containers | jersey-container-servlet | 2.40 |
org.glassfish.jersey.containers | jersey-container-servlet-core | 2.40 |
org.glassfish.jersey.core | jersey-client | 2.40 |
org.glassfish.jersey.core | jersey-common | 2.40 |
org.glassfish.jersey.core | jersey-server | 2.40 |
org.glassfish.jersey.inject | jersey-hk2 | 2.40 |
org.hibernate.validator | hibernate-validator | 6.1.7.Final |
org.ini4j | ini4j | 0.5.4 |
org.javassist | javassist | 3.29.2-GA |
org.jboss.logging | jboss-logging | 3.3.2.Final |
org.jdbi | jdbi | 2.63.1 |
org.jetbrains | annotations | 17.0.0 |
org.joda | joda-convert | 1.7 |
org.jodd | jodd-core | 3.5.2 |
org.json4s | json4s-ast_2.12 | 3.7.0-M11 |
org.json4s | json4s-core_2.12 | 3.7.0-M11 |
org.json4s | json4s-jackson_2.12 | 3.7.0-M11 |
org.json4s | json4s-scalap_2.12 | 3.7.0-M11 |
org.lz4 | lz4-java | 1.8.0 |
org.mlflow | mlflow-spark_2.12 | 2.9.1 |
org.objenesis | objenesis | 2.5.1 |
org.postgresql | postgresql | 42.6.1 |
org.roaringbitmap | RoaringBitmap | 0.9.45-databricks |
org.roaringbitmap | shims | 0.9.45-databricks |
org.rocksdb | rocksdbjni | 9.2.1 |
org.rosuda.REngine | REngine | 2.1.0 |
org.scala-lang | scala-compiler_2.12 | 2.12.15 |
org.scala-lang | scala-library_2.12 | 2.12.15 |
org.scala-lang | scala-reflect_2.12 | 2.12.15 |
org.scala-lang.modules | scala-collection-compat_2.12 | 2.11.0 |
org.scala-lang.modules | scala-java8-compat_2.12 | 0.9.1 |
org.scala-lang.modules | scala-parser-combinators_2.12 | 1.1.2 |
org.scala-lang.modules | scala-xml_2.12 | 1.2.0 |
org.scala-sbt | test-interface | 1.0 |
org.scalacheck | scalacheck_2.12 | 1.14.2 |
org.scalactic | scalactic_2.12 | 3.2.16 |
org.scalanlp | breeze-macros_2.12 | 2.1.0 |
org.scalanlp | breeze_2.12 | 2.1.0 |
org.scalatest | scalatest-compatible | 3.2.16 |
org.scalatest | scalatest-core_2.12 | 3.2.16 |
org.scalatest | scalatest-diagrams_2.12 | 3.2.16 |
org.scalatest | scalatest-featurespec_2.12 | 3.2.16 |
org.scalatest | scalatest-flatspec_2.12 | 3.2.16 |
org.scalatest | scalatest-freespec_2.12 | 3.2.16 |
org.scalatest | scalatest-funspec_2.12 | 3.2.16 |
org.scalatest | scalatest-funsuite_2.12 | 3.2.16 |
org.scalatest | scalatest-matchers-core_2.12 | 3.2.16 |
org.scalatest | scalatest-mustmatchers_2.12 | 3.2.16 |
org.scalatest | scalatest-propspec_2.12 | 3.2.16 |
org.scalatest | scalatest-refspec_2.12 | 3.2.16 |
org.scalatest | scalatest-shouldmatchers_2.12 | 3.2.16 |
org.scalatest | scalatest-wordspec_2.12 | 3.2.16 |
org.scalatest | scalatest_2.12 | 3.2.16 |
org.slf4j | jcl-over-slf4j | 2.0.7 |
org.slf4j | jul-to-slf4j | 2.0.7 |
org.slf4j | slf4j-api | 2.0.7 |
org.slf4j | slf4j-simple | 1.7.25 |
org.threeten | threeten-extra | 1.7.1 |
org.tukaani | xz | 1.9 |
org.typelevel | algebra_2.12 | 2.0.1 |
org.typelevel | cats-kernel_2.12 | 2.1.1 |
org.typelevel | spire-macros_2.12 | 0.17.0 |
org.typelevel | spire-platform_2.12 | 0.17.0 |
org.typelevel | spire-util_2.12 | 0.17.0 |
org.typelevel | spire_2.12 | 0.17.0 |
org.wildfly.openssl | wildfly-openssl | 1.1.3.Final |
org.xerial | sqlite-jdbc | 3.42.0.0 |
org.xerial.snappy | snappy-java | 1.1.10.3 |
org.yaml | snakeyaml | 2.0 |
oro | oro | 2.0.8 |
pl.edu.icm | JLargeArrays | 1.5 |
software.amazon.cryptools | AmazonCorrettoCryptoProvider | 1.6.2-linux-x86_64 |
stax | stax-api | 1.0.1 |