Databricks Runtime 13.1 (EoS)
Note
Support for this Databricks Runtime version has ended. For the end-of-support date, see End-of-support history. For all supported Databricks Runtime versions, see Databricks Runtime release notes versions and compatibility.
The following release notes provide information about Databricks Runtime 13.1, powered by Apache Spark 3.4.0.
Databricks released this version in May 2023.
New features and improvements
- Cluster support for JDK 17 (Public Preview)
- Add, change, or delete data in streaming tables
- Read Kafka with SQL
- New SQL built-in functions
- Unity Catalog support for cluster-scoped Python libraries
- Expanded default enablement for optimized writes in Unity Catalog
- Advanced support for stateful operators in Structured Streaming workloads
- Delta clone for Unity Catalog is in Public Preview
- Pub/Sub support for Structured Streaming
- Drop duplicates within watermarks in Structured Streaming
- Expanded support for Delta conversions from Iceberg tables with truncated partition columns
- Stream schema changes with column mapping in Delta Lake
- Remove START VERSION
- New H3 expressions available with Python
Cluster support for JDK 17 (Public Preview)
Databricks now provides cluster support for Java Development Kit (JDK) 17. See Databricks SDK for Java.
Add, change, or delete data in streaming tables
You can now use DML statements to modify streaming tables published to Unity Catalog by Delta Live Tables pipelines. See Add, change, or delete data in a streaming table and Add, change, or delete data in a target streaming table. You can also use DML statements to modify streaming tables created in Databricks SQL.
Read Kafka with SQL
You can now use the read_kafka SQL function for reading Kafka data. Streaming with SQL is supported only in Delta Live Tables or with streaming tables in Databricks SQL. See read_kafka table-valued function.
New SQL built-in functions
The following functions have been added:
- array_prepend(array, elem)
Returns
arrayprepended byelem. - try_aes_decrypt(expr, key [, mode [, padding]])
Decrypts a binary produced using AES encryption, and returns
NULLif there is an error. - sql_keywords() Returns a table of Azure Databricks SQL keywords.
Unity Catalog support for cluster-scoped Python libraries
Unity Catalog has some limitations on library usage. On Databricks Runtime 13.1 and above, cluster-scoped Python libraries are supported, including Python wheel files that are uploaded as workspace files. Libraries that are referenced using DBFS filepaths are not supported, whether in the DBFS root or an external location mounted to DBFS. Non-Python libraries are not supported. See Cluster libraries.
On Databricks Runtime 13.0 and below, cluster-scoped libraries are not supported on clusters that use shared access mode in a Unity Catalog-enabled workspace.
Expanded default enablement for optimized writes in Unity Catalog
Default optimized write support for Delta tables registered in Unity Catalog has expanded to include CTAS statements and INSERT operations for partitioned tables. This behavior aligns to defaults on SQL warehouses. See Optimized writes for Delta Lake on Azure Databricks.
Advanced support for stateful operators in Structured Streaming workloads
You can now chain multiple stateful operators together, meaning that you can feed the output of an operation such as a windowed aggregation to another stateful operation such as a join. See What is stateful streaming?.
Delta clone for Unity Catalog is in Public Preview
You can now use shallow clone to create new Unity Catalog managed tables from existing Unity Catalog managed tables. See Shallow clone for Unity Catalog tables.
Pub/Sub support for Structured Streaming
You can now use a built-in connector to subscribe to Google Pub/Sub with Structured Streaming. See Subscribe to Google Pub/Sub.
Drop duplicates within watermarks in Structured Streaming
You can now use dropDuplicatesWithinWatermark in combination with a specified watermark threshold to deduplicate records in Structured Streaming. See Drop duplicates within watermark.
Expanded support for Delta conversions from Iceberg tables with truncated partition columns
You can now use CLONE and CONVERT TO DELTA with Iceberg tables that have partitions defined on truncated columns of types int, long, and string. Truncated columns of type decimal are not supported.
Stream schema changes with column mapping in Delta Lake
You now can provide a schema tracking location to enable streaming from Delta tables with column mapping enabled. See Streaming with column mapping and schema changes.
Remove START VERSION
START VERSION is now deprecated for ALTER SHARE.
New H3 expressions available with Python
The h3_coverash3 and h3_coverash3string expressions are available with Python.
Bug fixes
Parquet failOnUnknownFields no longer silently drop data on type mismatch
If a Parquet file was read with just the failOnUnknownFields option or with Auto Loader in the failOnNewColumns schema evolution mode, columns that have different data types now fail and recommend using rescuedDataColumn. Auto Loader now correctly reads and no longer rescues Integer, Short, or Byte types if one of these data types is provided. The Parquet file suggests one of the other two types.
Breaking changes
Upgrade sqlite-jdbc version to 3.42.0.0 to address CVE-2023-32697
Upgrade sqlite-jdbc version from 3.8.11.2 to 3.42.0.0. The APIs of version 3.42.0.0 are not fully compatible with 3.8.11.2. If using sqlite-jdbc in your code, check the sqlite-jdbc compatibility report for details. If you migrate to 13.1 and use sqlite, confirm your methods and return type in version 3.42.0.0.
Library upgrades
- Upgraded Python libraries:
- facets-overview from 1.0.2 to 1.0.3
- filelock from 3.10.7 to 3.12.0
- pyarrow from 7.0.0 to 8.0.0
- tenacity from 8.0.1 to 8.1.0
- Upgraded R libraries:
- Upgraded Java libraries:
- com.github.ben-manes.caffeine.caffeine from 2.3.4 to 2.9.3
- io.delta.delta-sharing-spark_2.12 from 0.6.8 to 0.6.4
- net.snowflake.snowflake-jdbc from 3.13.29 to 3.13.22
- org.checkerframework.checker-qual from 3.5.0 to 3.19.0
- org.scalactic.scalactic_2.12 from 3.0.8 to 3.2.15
- org.scalatest.scalatest_2.12 from 3.0.8 to 3.2.15
- org.xerial.sqlite-jdbc from 3.8.11.2 to 3.42.0.0
Apache Spark
Databricks Runtime 13.1 includes Apache Spark 3.4.0. This release includes all Spark fixes and improvements included in Databricks Runtime 13.0 (EoS), as well as the following additional bug fixes and improvements made to Spark:
- [SPARK-42719] [DBRRM-199][SC-131578] Revert “[SC-125225] `MapOutputTracker#getMap…
- [SPARK-39696] [DBRRM-166][SC-130056][CORE] Revert [SC-127830]/
- [SPARK-43331] [SC-130064][CONNECT] Add Spark Connect SparkSession.interruptAll
- [SPARK-43332] [SC-130051][CONNECT][PYTHON] Make it possible to extend ChannelBuilder for SparkConnectClient
- [SPARK-43323] [SC-129966][SQL][PYTHON] Fix DataFrame.toPandas with Arrow enabled to handle exceptions properly
- [SPARK-42940] [SC-129896][SS][CONNECT] Improve session management for streaming queries
- [SPARK-43032] [SC-125756] [CONNECT][SS] Add Streaming query manager
- [SPARK-16484] [SC-129975][SQL] Add support for Datasketches HllSketch
- [SPARK-43260] [SC-129281][PYTHON] Migrate the Spark SQL pandas arrow type errors into error class.
- [SPARK-41766] [SC-129964][CORE] Handle decommission request sent before executor registration
- [SPARK-43307] [SC-129971][PYTHON] Migrate PandasUDF value errors into error class
- [SPARK-43206] [SC-129903] [SS] [CONNECT] StreamingQuery exception() include stack trace
- [SPARK-43311] [SC-129905][SS] Add RocksDB state store provider memory management enhancements
- [SPARK-43237] [SC-129898][CORE] Handle null exception message in event log
- [SPARK-43320] [SC-129899][SQL][HIVE] Directly call Hive 2.3.9 API
- [SPARK-43270] [SC-129897][PYTHON] Implement
__dir__()inpyspark.sql.dataframe.DataFrameto include columns - [SPARK-43183] Revert “[SC-128938][SS] Introduce a new callback “…
- [SPARK-43143] [SC-129902] [SS] [CONNECT] Scala StreamingQuery awaitTermination()
- [SPARK-43257] [SC-129675][SQL] Replace the error class _LEGACY_ERROR_TEMP_2022 by an internal error
- [SPARK-43198] [SC-129470][CONNECT] Fix “Could not initialise class ammonite…” error when using filter
- [SPARK-43165] [SC-129777][SQL] Move canWrite to DataTypeUtils
- [SPARK-43298] [SC-129729][PYTHON][ML] predict_batch_udf with scalar input fails with batch size of one
- [SPARK-43298] [SC-129700]Revert “[PYTHON][ML] predict_batch_udf with scalar input fails with batch size of one”
- [SPARK-43052] [SC-129663][CORE] Handle stacktrace with null file name in event log
- [SPARK-43183] [SC-128938][SS] Introduce a new callback “onQueryIdle” to StreamingQueryListener
- [SPARK-43209] [SC-129190][CONNECT][PYTHON] Migrate Expression errors into error class
- [SPARK-42151] [SC-128754][SQL] Align UPDATE assignments with table attributes
- [SPARK-43134] [SC-129468] [CONNECT] [SS] JVM client StreamingQuery exception() API
- [SPARK-43298] [SC-129699][PYTHON][ML] predict_batch_udf with scalar input fails with batch size of one
- [SPARK-43248] [SC-129660][SQL] Unnecessary serialize/deserialize of Path on parallel gather partition stats
- [SPARK-43274] [SC-129464][SPARK-43275][PYTHON][CONNECT] Introduce
PySparkNotImplementedError - [SPARK-43146] [SC-128804][CONNECT][PYTHON] Implement eager evaluation for repr and repr_html
- [SPARK-42953] [SC-129469][Connect][Followup] Fix maven test build for Scala client UDF tests
- [SPARK-43144] [SC-129280] Scala Client DataStreamReader table() API
- [SPARK-43136] [SC-129358][CONNECT] Adding groupByKey + mapGroup + coGroup functions
- [SPARK-43156] [SC-129672][SC-128532][SQL] Fix
COUNT(*) is nullbug in correlated scalar subquery - [SPARK-43046] [SC-129110] [SS] [Connect] Implemented Python API dropDuplicatesWithinWatermark for Spark Connect
- [SPARK-43199] [SC-129467][SQL] Make InlineCTE idempotent
- [SPARK-43293] [SC-129657][SQL]
__qualified_access_onlyshould be ignored in normal columns - [SPARK-43276] [SC-129461][CONNECT][PYTHON] Migrate Spark Connect Window errors into error class
- [SPARK-43174] [SC-129109][SQL] Fix SparkSQLCLIDriver completer
- [SPARK-43084] [SC-128654] [SS] Add applyInPandasWithState support for spark connect
- [SPARK-43119] [SC-129040][SQL] Support Get SQL Keywords Dynamically Thru JDBC API and TVF
- [SPARK-43082] [SC-129112][CONNECT][PYTHON] Arrow-optimized Python UDFs in Spark Connect
- [SPARK-43085] [SC-128432][SQL] Support column DEFAULT assignment for multi-part table names
- [SPARK-43226] [LC-671] Define extractors for file-constant metadata
- [SPARK-43210] [SC-129189][CONNECT][PYTHON] Introduce
PySparkAssertionError - [SPARK-43214] [SC-129199][SQL] Post driver-side metrics for LocalTableScanExec/CommandResultExec
- [SPARK-43285] [SC-129347] Fix ReplE2ESuite consistently failing with JDK 17
- [SPARK-43268] [SC-129249][SQL] Use proper error classes when exceptions are constructed with a message
- [SPARK-43142] [SC-129299] Fix DSL expressions on attributes with special characters
- [SPARK-43129] [SC-128896] Scala core API for streaming Spark Connect
- [SPARK-43233] [SC-129250] [SS] Add logging for Kafka Batch Reading for topic partition, offset range and task ID
- [SPARK-43249] [SC-129195][CONNECT] Fix missing stats for SQL Command
- [SPARK-42945] [SC-129188][CONNECT] Support PYSPARK_JVM_STACKTRACE_ENABLED in Spark Connect
- [SPARK-43178] [SC-129197][CONNECT][PYTHON] Migrate UDF errors into PySpark error framework
- [SPARK-43123] [SC-128494][SQL] Internal field metadata should not be leaked to catalogs
- [SPARK-43217] [SC-129205] Correctly recurse in nested maps/arrays in findNestedField
- [SPARK-43243] [SC-129294][PYTHON][CONNECT] Add level param to printSchema for Python
- [SPARK-43230] [SC-129191][CONNECT] Simplify
DataFrameNaFunctions.fillna - [SPARK-43088] [SC-128403][SQL] Respect RequiresDistributionAndOrdering in CTAS/RTAS
- [SPARK-43234] [SC-129192][CONNECT][PYTHON] Migrate
ValueErrorfrom Conect DataFrame into error class - [SPARK-43212] [SC-129187][SS][PYTHON] Migrate Structured Streaming errors into error class
- [SPARK-43239] [SC-129186][PS] Remove
null_countsfrom info() - [SPARK-43190] [SC-128930][SQL] ListQuery.childOutput should be consistent with child output
- [SPARK-43191] [SC-128924][CORE] Replace reflection w/ direct calling for Hadoop CallerContext
- [SPARK-43193] [SC-129042][SS] Remove workaround for HADOOP-12074
- [SPARK-42657] [SC-128621][CONNECT] Support to find and transfer client-side REPL classfiles to server as artifacts
- [SPARK-43098] [SC-77059][SQL] Fix correctness COUNT bug when scalar subquery has group by clause
- [SPARK-43213] [SC-129062][PYTHON] Add
DataFrame.offsetto vanilla PySpark - [SPARK-42982] [SC-128400][CONNECT][PYTHON] Fix createDataFrame to respect the given schema ddl
- [SPARK-43124] [SC-129011][SQL] Dataset.show projects CommandResults locally
- [SPARK-42998] [SC-127422][CONNECT][PYTHON] Fix DataFrame.collect with null struct
- [SPARK-41498] [SC-125343]Revert ” Propagate metadata through Union”
- [SPARK-42960] [SC-129010] [CONNECT] [SS] Add await_termination() and exception() API for Streaming Query in Python
- [SPARK-42552] [SC-128824][SQL] Correct the two-stage parsing strategy of antlr parser
- [SPARK-43207] [SC-128937][CONNECT] Add helper functions to extract value from literal expression
- [SPARK-43186] [SC-128841][SQL][HIVE] Remove workaround for FileSinkDesc
- [SPARK-43107] [SC-128533][SQL] Coalesce buckets in join applied on broadcast join stream side
- [SPARK-43195] [SC-128922][CORE] Remove unnecessary serializable wrapper in HadoopFSUtils
- [SPARK-43137] [SC-128828][SQL] Improve ArrayInsert if the position is foldable and positive.
- [SPARK-37829] [SC-128827][SQL] Dataframe.joinWith outer-join should return a null value for unmatched row
- [SPARK-43042] [SC-128602] [SS] [Connect] Add table() API support for DataStreamReader
- [SPARK-43153] [SC-128753][CONNECT] Skip Spark execution when the dataframe is local
- [SPARK-43064] [SC-128496][SQL] Spark SQL CLI SQL tab should only show once statement once
- [SPARK-43126] [SC-128447][SQL] Mark two Hive UDF expressions as stateful
- [SPARK-43111] [SC-128750][PS][CONNECT][PYTHON] Merge nested
ifstatements into singleifstatements - [SPARK-43113] [SC-128749][SQL] Evaluate stream-side variables when generating code for a bound condition
- [SPARK-42895] [SC-127258][CONNECT] Improve error messages for stopped Spark sessions
- [SPARK-42884] [SC-126662][CONNECT] Add Ammonite REPL integration
- [SPARK-43168] [SC-128674][SQL] Remove get PhysicalDataType method from Datatype class
- [SPARK-43121] [SC-128455][SQL] Use
BytesWritable.copyBytesinstead of manual copy in `HiveInspectors - [SPARK-42916] [SC-128389][SQL] JDBCTableCatalog Keeps Char/Varchar meta on the read-side
- [SPARK-43050] [SC-128550][SQL] Fix construct aggregate expressions by replacing grouping functions
- [SPARK-43095] [SC-128549][SQL] Avoid Once strategy’s idempotence is broken for batch:
Infer Filters - [SPARK-43130] [SC-128597][SQL] Move InternalType to PhysicalDataType
- [SPARK-43105] [SC-128456][CONNECT] Abbreviate Bytes and Strings in proto message
- [SPARK-43099] [SC-128596][SQL] Use
getNameinstead ofgetCanonicalNameto get builder class name when registering udf to FunctionRegistry - [SPARK-42994] [SC-128586][ML][CONNECT] PyTorch Distributor support Local Mode
- [SPARK-42859] Revert “[SC-127935][CONNECT][PS] Basic support for pandas API on Spark Connect”
- [SPARK-43021] [SC-128472][SQL]
CoalesceBucketsInJoinnot work when using AQE - [SPARK-43125] [SC-128477][CONNECT] Fix Connect Server Can’t Handle Exception With Null Message
- [SPARK-43147] [SC-128594] fix flake8 lint for local check
- [SPARK-43031] [SC-128360] [SS] [Connect] Enable unit test and doctest for streaming
- [SPARK-43039] [LC-67] Support custom fields in the file source _metadata column.
- [SPARK-43120] [SC-128407][SS] Add support for tracking pinned blocks memory usage for RocksDB state store
- [SPARK-43110] [SC-128381][SQL] Move asIntegral to PhysicalDataType
- [SPARK-43118] [SC-128398][SS] Remove unnecessary assert for UninterruptibleThread in KafkaMicroBatchStream
- [SPARK-43055] [SC-128331][CONNECT][PYTHON] Support duplicated nested field names
- [SPARK-42437] [SC-128339][PYTHON][CONNECT] PySpark catalog.cacheTable will allow to specify storage level
- [SPARK-42985] [SC-128332][CONNECT][PYTHON] Fix createDataFrame to respect the SQL configs
- [SPARK-39696] [SC-127830][CORE] Fix data race in access to TaskMetrics.externalAccums
- [SPARK-43103] [SC-128335][SQL] Moving Integral to PhysicalDataType
- [SPARK-42741] [SC-125547][SQL] Do not unwrap casts in binary comparison when literal is null
- [SPARK-43057] [SC-127948][CONNECT][PYTHON] Migrate Spark Connect Column errors into error class
- [SPARK-42859] [SC-127935][CONNECT][PS] Basic support for pandas API on Spark Connect
- [SPARK-43013] [SC-127773][PYTHON] Migrate
ValueErrorfrom DataFrame intoPySparkValueError. - [SPARK-43089] [SC-128051][CONNECT] Redact debug string in UI
- [SPARK-43028] [SC-128070][SQL] Add error class SQL_CONF_NOT_FOUND
- [SPARK-42999] [SC-127842][Connect] Dataset#foreach, foreachPartition
- [SPARK-43066] [SC-127937][SQL] Add test for dropDuplicates in JavaDatasetSuite
- [SPARK-43075] [SC-127939][CONNECT] Change
gRPCtogrpciowhen it is not installed. - [SPARK-42953] [SC-127809][Connect] Typed filter, map, flatMap, mapPartitions
- [SPARK-42597] [SC-125506][SQL] Support unwrap date type to timestamp type
- [SPARK-42931] [SC-127933][SS] Introduce dropDuplicatesWithinWatermark
- [SPARK-43073] [SC-127943][CONNECT] Add proto data types constants
- [SPARK-43077] [SC-128050][SQL] Improve the error message of UNRECOGNIZED_SQL_TYPE
- [SPARK-42951] [SC-128030][SS][Connect] DataStreamReader APIs
- [SPARK-43049] [SC-127846][SQL] Use CLOB instead of VARCHAR(255) for StringType for Oracle JDBC
- [SPARK-43018] [SC-127762][SQL] Fix bug for INSERT commands with timestamp literals
- [SPARK-42855] [SC-127722][SQL] Use runtime null checks in TableOutputResolver
- [SPARK-43030] [SC-127847][SQL] Deduplicate relations with metadata columns
- [SPARK-42993] [SC-127829][ML][CONNECT] Make PyTorch Distributor compatible with Spark Connect
- [SPARK-43058] [SC-128072][SQL] Move Numeric and Fractional to PhysicalDataType
- [SPARK-43056] [SC-127946][SS] RocksDB state store commit should continue background work only if its paused
- [SPARK-43059] [SC-127947][CONNECT][PYTHON] Migrate TypeError from DataFrame(Reader|Writer) into error class
- [SPARK-43071] [SC-128018][SQL] Support SELECT DEFAULT with ORDER BY, LIMIT, OFFSET for INSERT source relation
- [SPARK-43061] [SC-127956][CORE][SQL] Introduce PartitionEvaluator for SQL operator execution
- [SPARK-43067] [SC-127938][SS] Correct the location of error class resource file in Kafka connector
- [SPARK-43019] [SC-127844][SQL] Move Ordering to PhysicalDataType
- [SPARK-43010] [SC-127759][PYTHON] Migrate Column errors into error class
- [SPARK-42840] [SC-127782][SQL] Change
_LEGACY_ERROR_TEMP_2004error to internal error - [SPARK-43041] [SC-127765][SQL] Restore constructors of exceptions for compatibility in connector API
- [SPARK-42939] [SC-127761][SS][CONNECT] Core streaming Python API for Spark Connect
- [SPARK-42844] [SC-127766][SQL] Update the error class
_LEGACY_ERROR_TEMP_2008toINVALID_URL - [SPARK-42316] [SC-127720][SQL] Assign name to _LEGACY_ERROR_TEMP_2044
- [SPARK-42995] [SC-127723][CONNECT][PYTHON] Migrate Spark Connect DataFrame errors into error class
- [SPARK-42983] [SC-127717][CONNECT][PYTHON] Fix createDataFrame to handle 0-dim numpy array properly
- [SPARK-42955] [SC-127476][SQL] Skip classifyException and wrap AnalysisException for SparkThrowable
- [SPARK-42949] [SC-127255][SQL] Simplify code for NAAJ
- [SPARK-43011] [SC-127577][SQL]
array_insertshould fail with 0 index - [SPARK-42974] [SC-127487][CORE] Restore
Utils.createTempDirto use theShutdownHookManagerand clean upJavaUtils.createTempDirmethod. - [SPARK-42964] [SC-127585][SQL] PosgresDialect ‘42P07’ also means table already exists
- [SPARK-42978] [SC-127351][SQL] Derby&PG: RENAME cannot qualify a new-table-Name with a schema-Name
- [SPARK-37980] [SC-127668][SQL] Access row_index via _metadata if possible in tests
- [SPARK-42655] [SC-127591][SQL] Incorrect ambiguous column reference error
- [SPARK-43009] [SC-127596][SQL] Parameterized
sql()withAnyconstants - [SPARK-43026] [SC-127590][SQL] Apply AQE with non-exchange table cache
- [SPARK-42963] [SC-127576][SQL] Extend SparkSessionExtensions to inject rules into AQE query stage optimizer
- [SPARK-42918] [SC-127357] Generalize handling of metadata attributes in FileSourceStrategy
- [SPARK-42806] [SC-127452][SPARK-42811][CONNECT] Add
Catalogsupport - [SPARK-42997] [SC-127535][SQL] TableOutputResolver must use correct column paths in error messages for arrays and maps
- [SPARK-43006] [SC-127486][PYSPARK] Fix typo in StorageLevel eq()
- [SPARK-43005] [SC-127485][PYSPARK] Fix typo in pyspark/pandas/config.py
- [SPARK-43004] [SC-127457][CORE] Fix typo in ResourceRequest.equals()
- [SPARK-42907] [SC-126984][CONNECT][PYTHON] Implement Avro functions
- [SPARK-42979] [SC-127272][SQL] Define literal constructors as keywords
- [SPARK-42946] [SC-127252][SQL] Redact sensitive data which is nested by variable substitution
- [SPARK-42952] [SC-127260][SQL] Simplify the parameter of analyzer rule PreprocessTableCreation and DataSourceAnalysis
- [SPARK-42683] [LC-75] Automatically rename conflicting metadata columns
- [SPARK-42853] [SC-126101][FollowUp] Fix conflicts
- [SPARK-42929] [SC-126748][CONNECT] make mapInPandas / mapInArrow support “is_barrier”
- [SPARK-42968] [SC-127271][SS] Add option to skip commit coordinator as part of StreamingWrite API for DSv2 sources/sinks
- [SPARK-42954] [SC-127261][PYTHON][CONNECT] Add
YearMonthIntervalTypeto PySpark and Spark Connect Python Client - [SPARK-41359] [SC-127256][SQL] Use
PhysicalDataTypeinstead of DataType in UnsafeRow - [SPARK-42873] [SC-127262][SQL] Define Spark SQL types as keywords
- [SPARK-42808] [SC-126302][CORE] Avoid getting availableProcessors every time in
MapOutputTrackerMaster#getStatistics - [SPARK-42937] [SC-126880][SQL]
PlanSubqueriesshould setInSubqueryExec#shouldBroadcastto true - [SPARK-42896] [SC-126729][SQL][PYTHON] Make
mapInPandas/mapInArrowsupport barrier mode execution - [SPARK-42874] [SC-126442][SQL] Enable new golden file test framework for analysis for all input files
- [SPARK-42922] [SC-126850][SQL] Move from Random to SecureRandom
- [SPARK-42753] [SC-126369] ReusedExchange refers to non-existent nodes
- [SPARK-40822] [SC-126274][SQL] Stable derived column aliases
- [SPARK-42908] [SC-126856][PYTHON] Raise RuntimeError when SparkContext is required but not initialized
- [SPARK-42779] [SC-126042][SQL] Allow V2 writes to indicate advisory shuffle partition size
- [SPARK-42914] [SC-126727][PYTHON] Reuse
transformUnregisteredFunctionforDistributedSequenceID. - [SPARK-42878] [SC-126882][CONNECT] The table API in DataFrameReader could also accept options
- [SPARK-42927] [SC-126883][CORE] Change the access scope of
o.a.spark.util.Iterators#sizetoprivate[util] - [SPARK-42943] [SC-126879][SQL] Use LONGTEXT instead of TEXT for StringType for effective length
- [SPARK-37677] [SC-126855][CORE] Unzip could keep file permissions
- [SPARK-42891] [13.x][SC-126458][CONNECT][PYTHON] Implement CoGrouped Map API
- [SPARK-41876] [SC-126849][CONNECT][PYTHON] Implement DataFrame.toLocalIterator
- [SPARK-42930] [SC-126761][CORE][SQL] Change the access scope of
ProtobufSerDerelated implementations toprivate[protobuf] - [SPARK-42819] [SC-125879][SS] Add support for setting max_write_buffer_number and write_buffer_size for RocksDB used in streaming
- [SPARK-42924] [SC-126737][SQL][CONNECT][PYTHON] Clarify the comment of parameterized SQL args
- [SPARK-42748] [SC-126455][CONNECT] Server-side Artifact Management
- [SPARK-42816] [SC-126365][CONNECT] Support Max Message size up to 128MB
- [SPARK-42850] [SC-126109][SQL] Remove duplicated rule CombineFilters in Optimizer
- [SPARK-42662] [SC-126355][CONNECT][PS] Add proto message for pandas API on Spark default index
- [SPARK-42720] [SC-126136][PS][SQL] Uses expression for distributed-sequence default index instead of plan
- [SPARK-42790] [SC-126174][SQL] Abstract the excluded method for better test for JDBC docker tests.
- [SPARK-42900] [SC-126473][CONNECT][PYTHON] Fix createDataFrame to respect inference and column names
- [SPARK-42917] [SC-126657][SQL] Correct getUpdateColumnNullabilityQuery for DerbyDialect
- [SPARK-42684] [SC-125157][SQL] v2 catalog should not allow column default value by default
- [SPARK-42861] [SC-126635][SQL] Use private[sql] instead of protected[sql] to avoid generating API doc
- [SPARK-42920] [SC-126728][CONNECT][PYTHON] Enable tests for UDF with UDT
- [SPARK-42791] [SC-126617][SQL] Create a new golden file test framework for analysis
- [SPARK-42911] [SC-126652][PYTHON] Introduce more basic exceptions
- [SPARK-42904] [SC-126634][SQL] Char/Varchar Support for JDBC Catalog
- [SPARK-42901] [SC-126459][CONNECT][PYTHON] Move
StorageLevelinto a separate file to avoid potentialfile recursively imports - [SPARK-42894] [SC-126451][CONNECT] Support
cache/persist/unpersist/storageLevelfor Spark connect jvm client - [SPARK-42792] [SC-125852][SS] Add support for WRITE_FLUSH_BYTES for RocksDB used in streaming stateful operators
- [SPARK-41233] [SC-126441][CONNECT][PYTHON] Add array_prepend to Spark Connect Python client
- [SPARK-42681] [SC-125149][SQL] Relax ordering constraint for ALTER TABLE ADD|REPLACE column descriptor
- [SPARK-42889] [SC-126367][CONNECT][PYTHON] Implement cache, persist, unpersist, and storageLevel
- [SPARK-42824] [SC-125985][CONNECT][PYTHON] Provide a clear error message for unsupported JVM attributes
- [SPARK-42340] [SC-126131][CONNECT][PYTHON] Implement Grouped Map API
- [SPARK-42892] [SC-126454][SQL] Move sameType and relevant methods out of DataType
- [SPARK-42827] [SC-126126][CONNECT] Support
functions#array_prependfor Scala connect client - [SPARK-42823] [SC-125987][SQL]
spark-sqlshell supports multipart namespaces for initialization - [SPARK-42817] [SC-125960][CORE] Logging the shuffle service name once in ApplicationMaster
- [SPARK-42786] [SC-126438][Connect] Typed Select
- [SPARK-42800] [SC-125868][CONNECT][PYTHON][ML] Implement ml function
{array_to_vector, vector_to_array} - [SPARK-42052] [SC-126439][SQL] Codegen Support for HiveSimpleUDF
- [SPARK-41233] [SC-126110][SQL][PYTHON] Add
array_prependfunction - [SPARK-42864] [SC-126268][ML][3.4] Make
IsotonicRegression.PointsAccumulatorprivate - [SPARK-42876] [SC-126281][SQL] DataType’s physicalDataType should be private[sql]
- [SPARK-42101] [SC-125437][SQL] Make AQE support InMemoryTableScanExec
- [SPARK-41290] [SC-124030][SQL] Support GENERATED ALWAYS AS expressions for columns in create/replace table statements
- [SPARK-42870] [SC-126220][CONNECT] Move
toCatalystValuetoconnect-common - [SPARK-42247] [SC-126107][CONNECT][PYTHON] Fix UserDefinedFunction to have returnType
- [SPARK-42875] [SC-126258][CONNECT][PYTHON] Fix toPandas to handle timezone and map types properly
- [SPARK-42757] [SC-125626][CONNECT] Implement textFile for DataFrameReader
- [SPARK-42803] [SC-126081][CORE][SQL][ML] Use getParameterCount function instead of getParameterTypes.length
- [SPARK-42833] [SC-126043][SQL] Refactor
applyExtensionsinSparkSession - [SPARK-41765] Revert “[SC-123550][SQL] Pull out v1 write metrics…
- [SPARK-42848] [SC-126105][CONNECT][PYTHON] Implement DataFrame.registerTempTable
- [SPARK-42020] [SC-126103][CONNECT][PYTHON] Support UserDefinedType in Spark Connect
- [SPARK-42818] [SC-125861][CONNECT][PYTHON] Implement DataFrameReader/Writer.jdbc
- [SPARK-42812] [SC-125867][CONNECT] Add client_type to AddArtifactsRequest protobuf message
- [SPARK-42772] [SC-125860][SQL] Change the default value of JDBC options about push down to true
- [SPARK-42771] [SC-125855][SQL] Refactor HiveGenericUDF
- [SPARK-25050] [SC-123839][SQL] Avro: writing complex unions
- [SPARK-42765] [SC-125850][CONNECT][PYTHON] Enable importing
pandas_udffrompyspark.sql.connect.functions - [SPARK-42719] [SC-125225][CORE]
MapOutputTracker#getMapLocationshould respectspark.shuffle.reduceLocality.enabled - [SPARK-42480] [SC-125173][SQL] Improve the performance of drop partitions
- [SPARK-42689] [SC-125195][CORE][SHUFFLE] Allow ShuffleDriverComponent to declare if shuffle data is reliably stored
- [SPARK-42726] [SC-125279][CONNECT][PYTHON] Implement
DataFrame.mapInArrow - [SPARK-41765] [SC-123550][SQL] Pull out v1 write metrics to WriteFiles
- [SPARK-41171] [SC-124191][SQL] Infer and push down window limit through window if partitionSpec is empty
- [SPARK-42686] [SC-125292][CORE] Defer formatting for debug messages in TaskMemoryManager
- [SPARK-42756] [SC-125443][CONNECT][PYTHON] Helper function to convert proto literal to value in Python Client
- [SPARK-42793] [SC-125627][CONNECT]
connectmodule requiresbuild_profile_flags - [SPARK-42701] [SC-125192][SQL] Add the
try_aes_decrypt()function - [SPARK-42679] [SC-125438][CONNECT][PYTHON] createDataFrame doesn’t work with non-nullable schema
- [SPARK-42733] [SC-125542][CONNECT][Followup] Write without path or table
- [SPARK-42777] [SC-125525][SQL] Support converting TimestampNTZ catalog stats to plan stats
- [SPARK-42770] [SC-125558][CONNECT] Add
truncatedTo(ChronoUnit.MICROS)to makeSQLImplicitsTestSuitein Java 17 daily test GA task pass - [SPARK-42752] [SC-125550][PYSPARK][SQL] Make PySpark exceptions printable during initialization
- [SPARK-42732] [SC-125544][PYSPARK][CONNECT] Support spark connect session getActiveSession method
- [SPARK-42755] [SC-125442][CONNECT] Factor literal value conversion out to
connect-common - [SPARK-42747] [SC-125399][ML] Fix incorrect internal status of LoR and AFT
- [SPARK-42740] [SC-125439][SQL] Fix the bug that pushdown offset or paging is invalid for some built-in dialect
- [SPARK-42745] [SC-125332][SQL] Improved AliasAwareOutputExpression works with DSv2
- [SPARK-42743] [SC-125330][SQL] Support analyze TimestampNTZ columns
- [SPARK-42721] [SC-125371][CONNECT] RPC logging interceptor
- [SPARK-42691] [SC-125397][CONNECT][PYTHON] Implement Dataset.semanticHash
- [SPARK-42688] [SC-124922][CONNECT] Rename Connect proto Request client_id to session_id
- [SPARK-42310] [SC-122792][SQL] Assign name to _LEGACY_ERROR_TEMP_1289
- [SPARK-42685] [SC-125339][CORE] Optimize Utils.bytesToString routines
- [SPARK-42725] [SC-125296][CONNECT][PYTHON] Make LiteralExpression support array params
- [SPARK-42702] [SC-125293][SPARK-42623][SQL] Support parameterized query in subquery and CTE
- [SPARK-42697] [SC-125189][WEBUI] Fix /api/v1/applications to return total uptime instead of 0 for the duration field
- [SPARK-42733] [SC-125278][CONNECT][PYTHON] Fix DataFrameWriter.save to work without path parameter
- [SPARK-42376] [SC-124928][SS] Introduce watermark propagation among operators
- [SPARK-42710] [SC-125205][CONNECT][PYTHON] Rename FrameMap proto to MapPartitions
- [SPARK-37099] [SC-123542][SQL] Introduce the group limit of Window for rank-based filter to optimize top-k computation
- [SPARK-42630] [SC-125207][CONNECT][PYTHON] Introduce UnparsedDataType and delay parsing DDL string until SparkConnectClient is available
- [SPARK-42690] [SC-125193][CONNECT] Implement CSV/JSON parsing functions for Scala client
- [SPARK-42709] [SC-125172][PYTHON] Remove the assumption of
__file__being available - [SPARK-42318] [SC-122648][SPARK-42319][SQL] Assign name to LEGACY_ERROR_TEMP(2123|2125)
- [SPARK-42723] [SC-125183][SQL] Support parser data type json “timestamp_ltz” as TimestampType
- [SPARK-42722] [SC-125175][CONNECT][PYTHON] Python Connect def schema() should not cache the schema
- [SPARK-42643] [SC-125152][CONNECT][PYTHON] Register Java (aggregate) user-defined functions
- [SPARK-42656] [SC-125177][CONNECT][Followup] Fix the spark-connect script
- [SPARK-41516] [SC-123899] [SQL] Allow jdbc dialects to override the query used to create a table
- [SPARK-41725] [SC-124396][CONNECT] Eager Execution of DF.sql()
- [SPARK-42687] [SC-124896][SS] Better error message for the unsupport
pivotoperation in Streaming - [SPARK-42676] [SC-124809][SS] Write temp checkpoints for streaming queries to local filesystem even if default FS is set differently
- [SPARK-42303] [SC-122644][SQL] Assign name to _LEGACY_ERROR_TEMP_1326
- [SPARK-42553] [SC-124560][SQL] Ensure at least one time unit after “interval”
- [SPARK-42649] [SC-124576][CORE] Remove the standard Apache License header from the top of third-party source files
- [SPARK-42611] [SC-124395][SQL] Insert char/varchar length checks for inner fields during resolution
- [SPARK-42419] [SC-124019][CONNECT][PYTHON] Migrate into error framework for Spark Connect Column API.
- [SPARK-42637] [SC-124522][CONNECT] Add SparkSession.stop()
- [SPARK-42647] [SC-124647][PYTHON] Change alias for numpy deprecated and removed types
- [SPARK-42616] [SC-124389][SQL] SparkSQLCLIDriver shall only close started hive sessionState
- [SPARK-42593] [SC-124405][PS] Deprecate & remove the APIs that will be removed in pandas 2.0.
- [SPARK-41870] [SC-124402][CONNECT][PYTHON] Fix createDataFrame to handle duplicated column names
- [SPARK-42569] [SC-124379][CONNECT] Throw exceptions for unsupported session API
- [SPARK-42631] [SC-124526][CONNECT] Support custom extensions in Scala client
- [SPARK-41868] [SC-124387][CONNECT][PYTHON] Fix createDataFrame to support durations
- [SPARK-42572] [SC-124171][SQL][SS] Fix behavior for StateStoreProvider.validateStateRowFormat
Maintenance updates
See Databricks Runtime 13.1 maintenance updates.
System environment
- Operating System: Ubuntu 22.04.2 LTS
- Java: Zulu 8.70.0.23-CA-linux64
- Scala: 2.12.15
- Python: 3.10.12
- R: 4.2.2
- Delta Lake: 2.4.0
Installed Python libraries
| Library | Version | Library | Version | Library | Version |
|---|---|---|---|---|---|
| appdirs | 1.4.4 | argon2-cffi | 21.3.0 | argon2-cffi-bindings | 21.2.0 |
| asttokens | 2.2.1 | attrs | 21.4.0 | backcall | 0.2.0 |
| beautifulsoup4 | 4.11.1 | black | 22.6.0 | bleach | 4.1.0 |
| blinker | 1.4 | boto3 | 1.24.28 | botocore | 1.27.28 |
| certifi | 2022.9.14 | cffi | 1.15.1 | chardet | 4.0.0 |
| charset-normalizer | 2.0.4 | click | 8.0.4 | cryptography | 37.0.1 |
| cycler | 0.11.0 | Cython | 0.29.32 | dbus-python | 1.2.18 |
| debugpy | 1.5.1 | decorator | 5.1.1 | defusedxml | 0.7.1 |
| distlib | 0.3.6 | docstring-to-markdown | 0.12 | entrypoints | 0.4 |
| executing | 1.2.0 | facets-overview | 1.0.3 | fastjsonschema | 2.16.3 |
| filelock | 3.12.0 | fonttools | 4.25.0 | googleapis-common-protos | 1.56.4 |
| grpcio | 1.48.1 | grpcio-status | 1.48.1 | httplib2 | 0.20.2 |
| idna | 3.3 | importlib-metadata | 4.6.4 | ipykernel | 6.17.1 |
| ipython | 8.10.0 | ipython-genutils | 0.2.0 | ipywidgets | 7.7.2 |
| jedi | 0.18.1 | jeepney | 0.7.1 | Jinja2 | 2.11.3 |
| jmespath | 0.10.0 | joblib | 1.2.0 | jsonschema | 4.16.0 |
| jupyter-client | 7.3.4 | jupyter_core | 4.11.2 | jupyterlab-pygments | 0.1.2 |
| jupyterlab-widgets | 1.0.0 | keyring | 23.5.0 | kiwisolver | 1.4.2 |
| launchpadlib | 1.10.16 | lazr.restfulclient | 0.14.4 | lazr.uri | 1.0.6 |
| MarkupSafe | 2.0.1 | matplotlib | 3.5.2 | matplotlib-inline | 0.1.6 |
| mccabe | 0.7.0 | mistune | 0.8.4 | more-itertools | 8.10.0 |
| mypy-extensions | 0.4.3 | nbclient | 0.5.13 | nbconvert | 6.4.4 |
| nbformat | 5.5.0 | nest-asyncio | 1.5.5 | nodeenv | 1.7.0 |
| notebook | 6.4.12 | numpy | 1.21.5 | oauthlib | 3.2.0 |
| packaging | 21.3 | pandas | 1.4.4 | pandocfilters | 1.5.0 |
| parso | 0.8.3 | pathspec | 0.9.0 | patsy | 0.5.2 |
| pexpect | 4.8.0 | pickleshare | 0.7.5 | Pillow | 9.2.0 |
| pip | 22.2.2 | platformdirs | 2.5.2 | plotly | 5.9.0 |
| pluggy | 1.0.0 | prometheus-client | 0.14.1 | prompt-toolkit | 3.0.36 |
| protobuf | 3.19.4 | psutil | 5.9.0 | psycopg2 | 2.9.3 |
| ptyprocess | 0.7.0 | pure-eval | 0.2.2 | pyarrow | 8.0.0 |
| pycparser | 2.21 | pydantic | 1.10.6 | pyflakes | 3.0.1 |
| Pygments | 2.11.2 | PyGObject | 3.42.1 | PyJWT | 2.3.0 |
| pyodbc | 4.0.32 | pyparsing | 3.0.9 | pyright | 1.1.294 |
| pyrsistent | 0.18.0 | python-dateutil | 2.8.2 | python-lsp-jsonrpc | 1.0.0 |
| python-lsp-server | 1.7.1 | pytoolconfig | 1.2.2 | pytz | 2022.1 |
| pyzmq | 23.2.0 | requests | 2.28.1 | rope | 1.7.0 |
| s3transfer | 0.6.0 | scikit-learn | 1.1.1 | scipy | 1.9.1 |
| seaborn | 0.11.2 | SecretStorage | 3.3.1 | Send2Trash | 1.8.0 |
| setuptools | 63.4.1 | six | 1.16.0 | soupsieve | 2.3.1 |
| ssh-import-id | 5.11 | stack-data | 0.6.2 | statsmodels | 0.13.2 |
| tenacity | 8.1.0 | terminado | 0.13.1 | testpath | 0.6.0 |
| threadpoolctl | 2.2.0 | tokenize-rt | 4.2.1 | tomli | 2.0.1 |
| tornado | 6.1 | traitlets | 5.1.1 | typing_extensions | 4.3.0 |
| ujson | 5.4.0 | unattended-upgrades | 0.1 | urllib3 | 1.26.11 |
| virtualenv | 20.16.3 | wadllib | 1.3.6 | wcwidth | 0.2.5 |
| webencodings | 0.5.1 | whatthepatch | 1.0.2 | wheel | 0.37.1 |
| widgetsnbextension | 3.6.1 | yapf | 0.31.0 | zipp | 1.0.0 |
Installed R libraries
R libraries are installed from the Microsoft CRAN snapshot on 2023-02-10.
| Library | Version | Library | Version | Library | Version |
|---|---|---|---|---|---|
| arrow | 10.0.1 | askpass | 1.1 | assertthat | 0.2.1 |
| backports | 1.4.1 | base | 4.2.2 | base64enc | 0.1-3 |
| bit | 4.0.5 | bit64 | 4.0.5 | blob | 1.2.3 |
| boot | 1.3-28 | brew | 1.0-8 | brio | 1.1.3 |
| broom | 1.0.3 | bslib | 0.4.2 | cachem | 1.0.6 |
| callr | 3.7.3 | caret | 6.0-93 | cellranger | 1.1.0 |
| chron | 2.3-59 | class | 7.3-21 | cli | 3.6.0 |
| clipr | 0.8.0 | clock | 0.6.1 | cluster | 2.1.4 |
| codetools | 0.2-19 | colorspace | 2.1-0 | commonmark | 1.8.1 |
| compiler | 4.2.2 | config | 0.3.1 | cpp11 | 0.4.3 |
| crayon | 1.5.2 | credentials | 1.3.2 | curl | 5.0.0 |
| data.table | 1.14.6 | datasets | 4.2.2 | DBI | 1.1.3 |
| dbplyr | 2.3.0 | desc | 1.4.2 | devtools | 2.4.5 |
| diffobj | 0.3.5 | digest | 0.6.31 | downlit | 0.4.2 |
| dplyr | 1.1.0 | dtplyr | 1.2.2 | e1071 | 1.7-13 |
| ellipsis | 0.3.2 | evaluate | 0.20 | fansi | 1.0.4 |
| farver | 2.1.1 | fastmap | 1.1.0 | fontawesome | 0.5.0 |
| forcats | 1.0.0 | foreach | 1.5.2 | foreign | 0.8-82 |
| forge | 0.2.0 | fs | 1.6.1 | future | 1.31.0 |
| future.apply | 1.10.0 | gargle | 1.3.0 | generics | 0.1.3 |
| gert | 1.9.2 | ggplot2 | 3.4.0 | gh | 1.3.1 |
| gitcreds | 0.1.2 | glmnet | 4.1-6 | globals | 0.16.2 |
| glue | 1.6.2 | googledrive | 2.0.0 | googlesheets4 | 1.0.1 |
| gower | 1.0.1 | graphics | 4.2.2 | grDevices | 4.2.2 |
| grid | 4.2.2 | gridExtra | 2.3 | gsubfn | 0.7 |
| gtable | 0.3.1 | hardhat | 1.2.0 | haven | 2.5.1 |
| highr | 0.10 | hms | 1.1.2 | htmltools | 0.5.4 |
| htmlwidgets | 1.6.1 | httpuv | 1.6.8 | httr | 1.4.4 |
| ids | 1.0.1 | ini | 0.3.1 | ipred | 0.9-13 |
| isoband | 0.2.7 | iterators | 1.0.14 | jquerylib | 0.1.4 |
| jsonlite | 1.8.4 | KernSmooth | 2.23-20 | knitr | 1.42 |
| labeling | 0.4.2 | later | 1.3.0 | lattice | 0.20-45 |
| lava | 1.7.1 | lifecycle | 1.0.3 | listenv | 0.9.0 |
| lubridate | 1.9.1 | magrittr | 2.0.3 | markdown | 1.5 |
| MASS | 7.3-58.2 | Matrix | 1.5-1 | memoise | 2.0.1 |
| methods | 4.2.2 | mgcv | 1.8-41 | mime | 0.12 |
| miniUI | 0.1.1.1 | ModelMetrics | 1.2.2.2 | modelr | 0.1.10 |
| munsell | 0.5.0 | nlme | 3.1-162 | nnet | 7.3-18 |
| numDeriv | 2016.8-1.1 | openssl | 2.0.5 | parallel | 4.2.2 |
| parallelly | 1.34.0 | pillar | 1.8.1 | pkgbuild | 1.4.0 |
| pkgconfig | 2.0.3 | pkgdown | 2.0.7 | pkgload | 1.3.2 |
| plogr | 0.2.0 | plyr | 1.8.8 | praise | 1.0.0 |
| prettyunits | 1.1.1 | pROC | 1.18.0 | processx | 3.8.0 |
| prodlim | 2019.11.13 | profvis | 0.3.7 | progress | 1.2.2 |
| progressr | 0.13.0 | promises | 1.2.0.1 | proto | 1.0.0 |
| proxy | 0.4-27 | ps | 1.7.2 | purrr | 1.0.1 |
| r2d3 | 0.2.6 | R6 | 2.5.1 | ragg | 1.2.5 |
| randomForest | 4.7-1.1 | rappdirs | 0.3.3 | rcmdcheck | 1.4.0 |
| RColorBrewer | 1.1-3 | Rcpp | 1.0.10 | RcppEigen | 0.3.3.9.3 |
| readr | 2.1.3 | readxl | 1.4.2 | recipes | 1.0.4 |
| rematch | 1.0.1 | rematch2 | 2.1.2 | remotes | 2.4.2 |
| reprex | 2.0.2 | reshape2 | 1.4.4 | rlang | 1.0.6 |
| rmarkdown | 2.20 | RODBC | 1.3-20 | roxygen2 | 7.2.3 |
| rpart | 4.1.19 | rprojroot | 2.0.3 | Rserve | 1.8-12 |
| RSQLite | 2.2.20 | rstudioapi | 0.14 | rversions | 2.1.2 |
| rvest | 1.0.3 | sass | 0.4.5 | scales | 1.2.1 |
| selectr | 0.4-2 | sessioninfo | 1.2.2 | shape | 1.4.6 |
| shiny | 1.7.4 | sourcetools | 0.1.7-1 | sparklyr | 1.7.9 |
| SparkR | 3.4.0 | spatial | 7.3-15 | splines | 4.2.2 |
| sqldf | 0.4-11 | SQUAREM | 2021.1 | stats | 4.2.2 |
| stats4 | 4.2.2 | stringi | 1.7.12 | stringr | 1.5.0 |
| survival | 3.5-3 | sys | 3.4.1 | systemfonts | 1.0.4 |
| tcltk | 4.2.2 | testthat | 3.1.6 | textshaping | 0.3.6 |
| tibble | 3.1.8 | tidyr | 1.3.0 | tidyselect | 1.2.0 |
| tidyverse | 1.3.2 | timechange | 0.2.0 | timeDate | 4022.108 |
| tinytex | 0.44 | tools | 4.2.2 | tzdb | 0.3.0 |
| urlchecker | 1.0.1 | usethis | 2.1.6 | utf8 | 1.2.3 |
| utils | 4.2.2 | uuid | 1.1-0 | vctrs | 0.5.2 |
| viridisLite | 0.4.1 | vroom | 1.6.1 | waldo | 0.4.0 |
| whisker | 0.4.1 | withr | 2.5.0 | xfun | 0.37 |
| xml2 | 1.3.3 | xopen | 1.0.0 | xtable | 1.8-4 |
| yaml | 2.3.7 | zip | 2.2.2 |
Installed Java and Scala libraries (Scala 2.12 cluster version)
| Group ID | Artifact ID | Version |
|---|---|---|
| antlr | antlr | 2.7.7 |
| com.amazonaws | amazon-kinesis-client | 1.12.0 |
| com.amazonaws | aws-java-sdk-autoscaling | 1.12.390 |
| com.amazonaws | aws-java-sdk-cloudformation | 1.12.390 |
| com.amazonaws | aws-java-sdk-cloudfront | 1.12.390 |
| com.amazonaws | aws-java-sdk-cloudhsm | 1.12.390 |
| com.amazonaws | aws-java-sdk-cloudsearch | 1.12.390 |
| com.amazonaws | aws-java-sdk-cloudtrail | 1.12.390 |
| com.amazonaws | aws-java-sdk-cloudwatch | 1.12.390 |
| com.amazonaws | aws-java-sdk-cloudwatchmetrics | 1.12.390 |
| com.amazonaws | aws-java-sdk-codedeploy | 1.12.390 |
| com.amazonaws | aws-java-sdk-cognitoidentity | 1.12.390 |
| com.amazonaws | aws-java-sdk-cognitosync | 1.12.390 |
| com.amazonaws | aws-java-sdk-config | 1.12.390 |
| com.amazonaws | aws-java-sdk-core | 1.12.390 |
| com.amazonaws | aws-java-sdk-datapipeline | 1.12.390 |
| com.amazonaws | aws-java-sdk-directconnect | 1.12.390 |
| com.amazonaws | aws-java-sdk-directory | 1.12.390 |
| com.amazonaws | aws-java-sdk-dynamodb | 1.12.390 |
| com.amazonaws | aws-java-sdk-ec2 | 1.12.390 |
| com.amazonaws | aws-java-sdk-ecs | 1.12.390 |
| com.amazonaws | aws-java-sdk-efs | 1.12.390 |
| com.amazonaws | aws-java-sdk-elasticache | 1.12.390 |
| com.amazonaws | aws-java-sdk-elasticbeanstalk | 1.12.390 |
| com.amazonaws | aws-java-sdk-elasticloadbalancing | 1.12.390 |
| com.amazonaws | aws-java-sdk-elastictranscoder | 1.12.390 |
| com.amazonaws | aws-java-sdk-emr | 1.12.390 |
| com.amazonaws | aws-java-sdk-glacier | 1.12.390 |
| com.amazonaws | aws-java-sdk-glue | 1.12.390 |
| com.amazonaws | aws-java-sdk-iam | 1.12.390 |
| com.amazonaws | aws-java-sdk-importexport | 1.12.390 |
| com.amazonaws | aws-java-sdk-kinesis | 1.12.390 |
| com.amazonaws | aws-java-sdk-kms | 1.12.390 |
| com.amazonaws | aws-java-sdk-lambda | 1.12.390 |
| com.amazonaws | aws-java-sdk-logs | 1.12.390 |
| com.amazonaws | aws-java-sdk-machinelearning | 1.12.390 |
| com.amazonaws | aws-java-sdk-opsworks | 1.12.390 |
| com.amazonaws | aws-java-sdk-rds | 1.12.390 |
| com.amazonaws | aws-java-sdk-redshift | 1.12.390 |
| com.amazonaws | aws-java-sdk-route53 | 1.12.390 |
| com.amazonaws | aws-java-sdk-s3 | 1.12.390 |
| com.amazonaws | aws-java-sdk-ses | 1.12.390 |
| com.amazonaws | aws-java-sdk-simpledb | 1.12.390 |
| com.amazonaws | aws-java-sdk-simpleworkflow | 1.12.390 |
| com.amazonaws | aws-java-sdk-sns | 1.12.390 |
| com.amazonaws | aws-java-sdk-sqs | 1.12.390 |
| com.amazonaws | aws-java-sdk-ssm | 1.12.390 |
| com.amazonaws | aws-java-sdk-storagegateway | 1.12.390 |
| com.amazonaws | aws-java-sdk-sts | 1.12.390 |
| com.amazonaws | aws-java-sdk-support | 1.12.390 |
| com.amazonaws | aws-java-sdk-swf-libraries | 1.11.22 |
| com.amazonaws | aws-java-sdk-workspaces | 1.12.390 |
| com.amazonaws | jmespath-java | 1.12.390 |
| com.clearspring.analytics | stream | 2.9.6 |
| com.databricks | Rserve | 1.8-3 |
| com.databricks | jets3t | 0.7.1-0 |
| com.databricks.scalapb | compilerplugin_2.12 | 0.4.15-10 |
| com.databricks.scalapb | scalapb-runtime_2.12 | 0.4.15-10 |
| com.esotericsoftware | kryo-shaded | 4.0.2 |
| com.esotericsoftware | minlog | 1.3.0 |
| com.fasterxml | classmate | 1.3.4 |
| com.fasterxml.jackson.core | jackson-annotations | 2.14.2 |
| com.fasterxml.jackson.core | jackson-core | 2.14.2 |
| com.fasterxml.jackson.core | jackson-databind | 2.14.2 |
| com.fasterxml.jackson.dataformat | jackson-dataformat-cbor | 2.14.2 |
| com.fasterxml.jackson.datatype | jackson-datatype-joda | 2.14.2 |
| com.fasterxml.jackson.datatype | jackson-datatype-jsr310 | 2.13.4 |
| com.fasterxml.jackson.module | jackson-module-paranamer | 2.14.2 |
| com.fasterxml.jackson.module | jackson-module-scala_2.12 | 2.14.2 |
| com.github.ben-manes.caffeine | caffeine | 2.9.3 |
| com.github.fommil | jniloader | 1.1 |
| com.github.fommil.netlib | native_ref-java | 1.1 |
| com.github.fommil.netlib | native_ref-java | 1.1-natives |
| com.github.fommil.netlib | native_system-java | 1.1 |
| com.github.fommil.netlib | native_system-java | 1.1-natives |
| com.github.fommil.netlib | netlib-native_ref-linux-x86_64 | 1.1-natives |
| com.github.fommil.netlib | netlib-native_system-linux-x86_64 | 1.1-natives |
| com.github.luben | zstd-jni | 1.5.2-5 |
| com.github.wendykierp | JTransforms | 3.1 |
| com.google.code.findbugs | jsr305 | 3.0.0 |
| com.google.code.gson | gson | 2.8.9 |
| com.google.crypto.tink | tink | 1.7.0 |
| com.google.errorprone | error_prone_annotations | 2.10.0 |
| com.google.flatbuffers | flatbuffers-java | 1.12.0 |
| com.google.guava | guava | 15.0 |
| com.google.protobuf | protobuf-java | 2.6.1 |
| com.h2database | h2 | 2.1.214 |
| com.helger | profiler | 1.1.1 |
| com.jcraft | jsch | 0.1.55 |
| com.jolbox | bonecp | 0.8.0.RELEASE |
| com.lihaoyi | sourcecode_2.12 | 0.1.9 |
| com.microsoft.azure | azure-data-lake-store-sdk | 2.3.9 |
| com.microsoft.sqlserver | mssql-jdbc | 11.2.2.jre8 |
| com.ning | compress-lzf | 1.1.2 |
| com.sun.mail | javax.mail | 1.5.2 |
| com.sun.xml.bind | jaxb-core | 2.2.11 |
| com.sun.xml.bind | jaxb-impl | 2.2.11 |
| com.tdunning | json | 1.8 |
| com.thoughtworks.paranamer | paranamer | 2.8 |
| com.trueaccord.lenses | lenses_2.12 | 0.4.12 |
| com.twitter | chill-java | 0.10.0 |
| com.twitter | chill_2.12 | 0.10.0 |
| com.twitter | util-app_2.12 | 7.1.0 |
| com.twitter | util-core_2.12 | 7.1.0 |
| com.twitter | util-function_2.12 | 7.1.0 |
| com.twitter | util-jvm_2.12 | 7.1.0 |
| com.twitter | util-lint_2.12 | 7.1.0 |
| com.twitter | util-registry_2.12 | 7.1.0 |
| com.twitter | util-stats_2.12 | 7.1.0 |
| com.typesafe | config | 1.2.1 |
| com.typesafe.scala-logging | scala-logging_2.12 | 3.7.2 |
| com.uber | h3 | 3.7.0 |
| com.univocity | univocity-parsers | 2.9.1 |
| com.zaxxer | HikariCP | 4.0.3 |
| commons-cli | commons-cli | 1.5.0 |
| commons-codec | commons-codec | 1.15 |
| commons-collections | commons-collections | 3.2.2 |
| commons-dbcp | commons-dbcp | 1.4 |
| commons-fileupload | commons-fileupload | 1.5 |
| commons-httpclient | commons-httpclient | 3.1 |
| commons-io | commons-io | 2.11.0 |
| commons-lang | commons-lang | 2.6 |
| commons-logging | commons-logging | 1.1.3 |
| commons-pool | commons-pool | 1.5.4 |
| dev.ludovic.netlib | arpack | 3.0.3 |
| dev.ludovic.netlib | blas | 3.0.3 |
| dev.ludovic.netlib | lapack | 3.0.3 |
| info.ganglia.gmetric4j | gmetric4j | 1.0.10 |
| io.airlift | aircompressor | 0.21 |
| io.delta | delta-sharing-spark_2.12 | 0.6.4 |
| io.dropwizard.metrics | metrics-core | 4.2.10 |
| io.dropwizard.metrics | metrics-graphite | 4.2.10 |
| io.dropwizard.metrics | metrics-healthchecks | 4.2.10 |
| io.dropwizard.metrics | metrics-jetty9 | 4.2.10 |
| io.dropwizard.metrics | metrics-jmx | 4.2.10 |
| io.dropwizard.metrics | metrics-json | 4.2.10 |
| io.dropwizard.metrics | metrics-jvm | 4.2.10 |
| io.dropwizard.metrics | metrics-servlets | 4.2.10 |
| io.netty | netty-all | 4.1.87.Final |
| io.netty | netty-buffer | 4.1.87.Final |
| io.netty | netty-codec | 4.1.87.Final |
| io.netty | netty-codec-http | 4.1.87.Final |
| io.netty | netty-codec-http2 | 4.1.87.Final |
| io.netty | netty-codec-socks | 4.1.87.Final |
| io.netty | netty-common | 4.1.87.Final |
| io.netty | netty-handler | 4.1.87.Final |
| io.netty | netty-handler-proxy | 4.1.87.Final |
| io.netty | netty-resolver | 4.1.87.Final |
| io.netty | netty-transport | 4.1.87.Final |
| io.netty | netty-transport-classes-epoll | 4.1.87.Final |
| io.netty | netty-transport-classes-kqueue | 4.1.87.Final |
| io.netty | netty-transport-native-epoll | 4.1.87.Final |
| io.netty | netty-transport-native-epoll | 4.1.87.Final-linux-aarch_64 |
| io.netty | netty-transport-native-epoll | 4.1.87.Final-linux-x86_64 |
| io.netty | netty-transport-native-kqueue | 4.1.87.Final-osx-aarch_64 |
| io.netty | netty-transport-native-kqueue | 4.1.87.Final-osx-x86_64 |
| io.netty | netty-transport-native-unix-common | 4.1.87.Final |
| io.prometheus | simpleclient | 0.7.0 |
| io.prometheus | simpleclient_common | 0.7.0 |
| io.prometheus | simpleclient_dropwizard | 0.7.0 |
| io.prometheus | simpleclient_pushgateway | 0.7.0 |
| io.prometheus | simpleclient_servlet | 0.7.0 |
| io.prometheus.jmx | collector | 0.12.0 |
| jakarta.annotation | jakarta.annotation-api | 1.3.5 |
| jakarta.servlet | jakarta.servlet-api | 4.0.3 |
| jakarta.validation | jakarta.validation-api | 2.0.2 |
| jakarta.ws.rs | jakarta.ws.rs-api | 2.1.6 |
| javax.activation | activation | 1.1.1 |
| javax.el | javax.el-api | 2.2.4 |
| javax.jdo | jdo-api | 3.0.1 |
| javax.transaction | jta | 1.1 |
| javax.transaction | transaction-api | 1.1 |
| javax.xml.bind | jaxb-api | 2.2.11 |
| javolution | javolution | 5.5.1 |
| jline | jline | 2.14.6 |
| joda-time | joda-time | 2.12.1 |
| ml.combust.mleap | mleap-databricks-runtime_2.12 | v0.20.0-db2 |
| net.java.dev.jna | jna | 5.8.0 |
| net.razorvine | pickle | 1.3 |
| net.sf.jpam | jpam | 1.1 |
| net.sf.opencsv | opencsv | 2.3 |
| net.sf.supercsv | super-csv | 2.2.0 |
| net.snowflake | snowflake-ingest-sdk | 0.9.6 |
| net.snowflake | snowflake-jdbc | 3.13.22 |
| net.sourceforge.f2j | arpack_combined_all | 0.1 |
| org.acplt.remotetea | remotetea-oncrpc | 1.1.2 |
| org.antlr | ST4 | 4.0.4 |
| org.antlr | antlr-runtime | 3.5.2 |
| org.antlr | antlr4-runtime | 4.9.3 |
| org.antlr | stringtemplate | 3.2.1 |
| org.apache.ant | ant | 1.9.16 |
| org.apache.ant | ant-jsch | 1.9.16 |
| org.apache.ant | ant-launcher | 1.9.16 |
| org.apache.arrow | arrow-format | 11.0.0 |
| org.apache.arrow | arrow-memory-core | 11.0.0 |
| org.apache.arrow | arrow-memory-netty | 11.0.0 |
| org.apache.arrow | arrow-vector | 11.0.0 |
| org.apache.avro | avro | 1.11.1 |
| org.apache.avro | avro-ipc | 1.11.1 |
| org.apache.avro | avro-mapred | 1.11.1 |
| org.apache.commons | commons-collections4 | 4.4 |
| org.apache.commons | commons-compress | 1.21 |
| org.apache.commons | commons-crypto | 1.1.0 |
| org.apache.commons | commons-lang3 | 3.12.0 |
| org.apache.commons | commons-math3 | 3.6.1 |
| org.apache.commons | commons-text | 1.10.0 |
| org.apache.curator | curator-client | 2.13.0 |
| org.apache.curator | curator-framework | 2.13.0 |
| org.apache.curator | curator-recipes | 2.13.0 |
| org.apache.datasketches | datasketches-java | 3.1.0 |
| org.apache.datasketches | datasketches-memory | 2.0.0 |
| org.apache.derby | derby | 10.14.2.0 |
| org.apache.hadoop | hadoop-client-runtime | 3.3.4 |
| org.apache.hive | hive-beeline | 2.3.9 |
| org.apache.hive | hive-cli | 2.3.9 |
| org.apache.hive | hive-jdbc | 2.3.9 |
| org.apache.hive | hive-llap-client | 2.3.9 |
| org.apache.hive | hive-llap-common | 2.3.9 |
| org.apache.hive | hive-serde | 2.3.9 |
| org.apache.hive | hive-shims | 2.3.9 |
| org.apache.hive | hive-storage-api | 2.8.1 |
| org.apache.hive.shims | hive-shims-0.23 | 2.3.9 |
| org.apache.hive.shims | hive-shims-common | 2.3.9 |
| org.apache.hive.shims | hive-shims-scheduler | 2.3.9 |
| org.apache.httpcomponents | httpclient | 4.5.14 |
| org.apache.httpcomponents | httpcore | 4.4.16 |
| org.apache.ivy | ivy | 2.5.1 |
| org.apache.logging.log4j | log4j-1.2-api | 2.19.0 |
| org.apache.logging.log4j | log4j-api | 2.19.0 |
| org.apache.logging.log4j | log4j-core | 2.19.0 |
| org.apache.logging.log4j | log4j-slf4j2-impl | 2.19.0 |
| org.apache.mesos | mesos | 1.11.0-shaded-protobuf |
| org.apache.orc | orc-core | 1.8.3-shaded-protobuf |
| org.apache.orc | orc-mapreduce | 1.8.3-shaded-protobuf |
| org.apache.orc | orc-shims | 1.8.3 |
| org.apache.thrift | libfb303 | 0.9.3 |
| org.apache.thrift | libthrift | 0.12.0 |
| org.apache.xbean | xbean-asm9-shaded | 4.22 |
| org.apache.yetus | audience-annotations | 0.13.0 |
| org.apache.zookeeper | zookeeper | 3.6.3 |
| org.apache.zookeeper | zookeeper-jute | 3.6.3 |
| org.checkerframework | checker-qual | 3.19.0 |
| org.codehaus.jackson | jackson-core-asl | 1.9.13 |
| org.codehaus.jackson | jackson-mapper-asl | 1.9.13 |
| org.codehaus.janino | commons-compiler | 3.0.16 |
| org.codehaus.janino | janino | 3.0.16 |
| org.datanucleus | datanucleus-api-jdo | 4.2.4 |
| org.datanucleus | datanucleus-core | 4.1.17 |
| org.datanucleus | datanucleus-rdbms | 4.1.19 |
| org.datanucleus | javax.jdo | 3.2.0-m3 |
| org.eclipse.jetty | jetty-client | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-continuation | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-http | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-io | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-jndi | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-plus | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-proxy | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-security | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-server | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-servlet | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-servlets | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-util | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-util-ajax | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-webapp | 9.4.50.v20221201 |
| org.eclipse.jetty | jetty-xml | 9.4.50.v20221201 |
| org.eclipse.jetty.websocket | websocket-api | 9.4.50.v20221201 |
| org.eclipse.jetty.websocket | websocket-client | 9.4.50.v20221201 |
| org.eclipse.jetty.websocket | websocket-common | 9.4.50.v20221201 |
| org.eclipse.jetty.websocket | websocket-server | 9.4.50.v20221201 |
| org.eclipse.jetty.websocket | websocket-servlet | 9.4.50.v20221201 |
| org.fusesource.leveldbjni | leveldbjni-all | 1.8 |
| org.glassfish.hk2 | hk2-api | 2.6.1 |
| org.glassfish.hk2 | hk2-locator | 2.6.1 |
| org.glassfish.hk2 | hk2-utils | 2.6.1 |
| org.glassfish.hk2 | osgi-resource-locator | 1.0.3 |
| org.glassfish.hk2.external | aopalliance-repackaged | 2.6.1 |
| org.glassfish.hk2.external | jakarta.inject | 2.6.1 |
| org.glassfish.jersey.containers | jersey-container-servlet | 2.36 |
| org.glassfish.jersey.containers | jersey-container-servlet-core | 2.36 |
| org.glassfish.jersey.core | jersey-client | 2.36 |
| org.glassfish.jersey.core | jersey-common | 2.36 |
| org.glassfish.jersey.core | jersey-server | 2.36 |
| org.glassfish.jersey.inject | jersey-hk2 | 2.36 |
| org.hibernate.validator | hibernate-validator | 6.1.7.Final |
| org.javassist | javassist | 3.25.0-GA |
| org.jboss.logging | jboss-logging | 3.3.2.Final |
| org.jdbi | jdbi | 2.63.1 |
| org.jetbrains | annotations | 17.0.0 |
| org.joda | joda-convert | 1.7 |
| org.jodd | jodd-core | 3.5.2 |
| org.json4s | json4s-ast_2.12 | 3.7.0-M11 |
| org.json4s | json4s-core_2.12 | 3.7.0-M11 |
| org.json4s | json4s-jackson_2.12 | 3.7.0-M11 |
| org.json4s | json4s-scalap_2.12 | 3.7.0-M11 |
| org.lz4 | lz4-java | 1.8.0 |
| org.mariadb.jdbc | mariadb-java-client | 2.7.4 |
| org.mlflow | mlflow-spark | 2.2.0 |
| org.objenesis | objenesis | 2.5.1 |
| org.postgresql | postgresql | 42.3.8 |
| org.roaringbitmap | RoaringBitmap | 0.9.39 |
| org.roaringbitmap | shims | 0.9.39 |
| org.rocksdb | rocksdbjni | 7.8.3 |
| org.rosuda.REngine | REngine | 2.1.0 |
| org.scala-lang | scala-compiler_2.12 | 2.12.15 |
| org.scala-lang | scala-library_2.12 | 2.12.15 |
| org.scala-lang | scala-reflect_2.12 | 2.12.15 |
| org.scala-lang.modules | scala-collection-compat_2.12 | 2.4.3 |
| org.scala-lang.modules | scala-parser-combinators_2.12 | 1.1.2 |
| org.scala-lang.modules | scala-xml_2.12 | 1.2.0 |
| org.scala-sbt | test-interface | 1.0 |
| org.scalacheck | scalacheck_2.12 | 1.14.2 |
| org.scalactic | scalactic_2.12 | 3.2.15 |
| org.scalanlp | breeze-macros_2.12 | 2.1.0 |
| org.scalanlp | breeze_2.12 | 2.1.0 |
| org.scalatest | scalatest-compatible | 3.2.15 |
| org.scalatest | scalatest-core_2.12 | 3.2.15 |
| org.scalatest | scalatest-diagrams_2.12 | 3.2.15 |
| org.scalatest | scalatest-featurespec_2.12 | 3.2.15 |
| org.scalatest | scalatest-flatspec_2.12 | 3.2.15 |
| org.scalatest | scalatest-freespec_2.12 | 3.2.15 |
| org.scalatest | scalatest-funspec_2.12 | 3.2.15 |
| org.scalatest | scalatest-funsuite_2.12 | 3.2.15 |
| org.scalatest | scalatest-matchers-core_2.12 | 3.2.15 |
| org.scalatest | scalatest-mustmatchers_2.12 | 3.2.15 |
| org.scalatest | scalatest-propspec_2.12 | 3.2.15 |
| org.scalatest | scalatest-refspec_2.12 | 3.2.15 |
| org.scalatest | scalatest-shouldmatchers_2.12 | 3.2.15 |
| org.scalatest | scalatest-wordspec_2.12 | 3.2.15 |
| org.scalatest | scalatest_2.12 | 3.2.15 |
| org.slf4j | jcl-over-slf4j | 2.0.6 |
| org.slf4j | jul-to-slf4j | 2.0.6 |
| org.slf4j | slf4j-api | 2.0.6 |
| org.threeten | threeten-extra | 1.7.1 |
| org.tukaani | xz | 1.9 |
| org.typelevel | algebra_2.12 | 2.0.1 |
| org.typelevel | cats-kernel_2.12 | 2.1.1 |
| org.typelevel | spire-macros_2.12 | 0.17.0 |
| org.typelevel | spire-platform_2.12 | 0.17.0 |
| org.typelevel | spire-util_2.12 | 0.17.0 |
| org.typelevel | spire_2.12 | 0.17.0 |
| org.wildfly.openssl | wildfly-openssl | 1.1.3.Final |
| org.xerial | sqlite-jdbc | 3.42.0.0 |
| org.xerial.snappy | snappy-java | 1.1.8.4 |
| org.yaml | snakeyaml | 1.33 |
| oro | oro | 2.0.8 |
| pl.edu.icm | JLargeArrays | 1.5 |
| software.amazon.cryptools | AmazonCorrettoCryptoProvider | 1.6.1-linux-x86_64 |
| software.amazon.ion | ion-java | 1.0.2 |
| stax | stax-api | 1.0.1 |