Muistiinpano
Tämän sivun käyttö edellyttää valtuutusta. Voit yrittää kirjautua sisään tai vaihtaa hakemistoa.
Tämän sivun käyttö edellyttää valtuutusta. Voit yrittää vaihtaa hakemistoa.
The following release notes provide information about Databricks Runtime 18.1 (Beta).
Azure Databricks released this version in February 2026.
Important
Databricks Runtime 18.1 is in Beta. The contents of the supported environments might change during the Beta. Changes can include the list of packages or versions of installed packages.
New features and improvements
- Auto Loader uses file events by default when available
- Optimized writes for partitioned Unity Catalog tables created with CRTAS
- DATETIMEOFFSET data type support for Microsoft Azure Synapse
- Google BigQuery table comments
- Schema evolution with INSERT statements
- Preserved NULL struct values in INSERT operations
- Delta Sharing multi-statement transaction support
- DataFrame checkpoints in volume paths
- SQL commands no longer re-run when calling cache()
Auto Loader uses file events by default when available
Auto Loader uses file events by default when loading from an external location with file events enabled, providing improved performance and cost over directory listing. See Auto Loader with file events overview. File events aren't used if your stream code sets useIncrementalListing or useNotifications. To use directory listing instead, set useManagedFileEvents to false.
Optimized writes for partitioned Unity Catalog tables created with CRTAS
Optimized writes now correctly apply to partitioned Unity Catalog tables created via CREATE OR REPLACE TABLE ... AS SELECT (CRTAS), producing fewer, better-sized files. This behavior is enabled by default. To disable it, set spark.databricks.delta.optimizeWrite.UCTableCRTAS.enabled to false.
DATETIMEOFFSET data type support for Microsoft Azure Synapse
The DATETIMEOFFSET data type is now supported for Microsoft Azure Synapse connections.
Google BigQuery table comments
Table descriptions for Google BigQuery are now resolved and exposed as table comments.
Schema evolution with INSERT statements
Use the WITH SCHEMA EVOLUTION clause with SQL INSERT statements to automatically evolve the target table's schema during insert operations. For example:
INSERT WITH SCHEMA EVOLUTION INTO students TABLE visiting_students_with_additional_id;
For more information, see Enable schema evolution.
Preserved NULL struct values in INSERT operations
INSERT operations with schema evolution or implicit casting now correctly preserve NULL struct values when the source and target tables have differing struct field orders.
Delta Sharing multi-statement transaction support
Delta Sharing tables that use pre-signed URL or cloud token sharing modes now support multi-statement transactions. On first access within a transaction, the table version is pinned and reused for all subsequent reads in that transaction, so reads stay consistent.
Limitations:
- Time travel, change data feed, and streaming aren't supported.
- Multi-statement transactions aren't supported on shared tables without history.
- Shared views and foreign entities aren't allowed on untrusted compute.
DataFrame checkpoints in volume paths
DataFrame checkpoints now support Unity Catalog volume paths. Configure the checkpoint path using SparkContext.setCheckpointDir on dedicated clusters or the spark.checkpoint.dir configuration for Standard Clusters.
SQL commands no longer re-run when calling cache()
SQL commands are no longer re-run when calling .cache() on the result DataFrame. This includes commands such as SHOW TABLES and SHOW NAMESPACES.
Behavioral changes
- FILTER clause for MEASURE aggregate functions
- Optimized writes for Unity Catalog CRTAS operations
- DataFrame cache refresh for fine-grained access control tables
- Timestamp partition values use session timezone
- DESCRIBE FLOW reserved keyword
- Geospatial operations performance improvement
- Exception types for SQLSTATE
- Automatic streaming type widening
FILTER clause for MEASURE aggregate functions
MEASURE aggregate functions now support FILTER clauses. Previously, filters were silently ignored.
Optimized writes for Unity Catalog CRTAS operations
CREATE OR REPLACE TABLE AS SELECT (CRTAS) operations on partitioned Unity Catalog tables now apply optimized writes by default. To disable, set spark.databricks.delta.optimizeWrite.UCTableCRTAS.enabled to false.
DataFrame cache refresh for fine-grained access control tables
Writing to fine-grained access control tables on dedicated compute now refreshes cached DataFrames that depend on the table.
Timestamp partition values use session timezone
Timestamp partition values use the Spark session timezone instead of the JVM timezone. If you have timestamp partitions written before Databricks Runtime 18.0, run SHOW PARTITIONS to verify your partition metadata before writing new data.
DESCRIBE FLOW reserved keyword
The DESCRIBE FLOW command is now available. If you have a table named flow, use DESCRIBE schema.flow, DESCRIBE TABLE flow, or DESCRIBE `flow` with backticks.
Geospatial operations performance improvement
Geospatial boolean set operations use a new implementation with improved performance. Minor precision differences might occur after the 15th decimal place.
Exception types for SQLSTATE
Exception types are updated to support SQLSTATE. If your code parses exceptions by string matching or catches specific exception types, update your error handling logic.
Automatic streaming type widening
Streaming reads on Delta tables automatically handle column type widening. To require manual acknowledgement, set spark.databricks.delta.typeWidening.enableStreamingSchemaTracking to true.
Library upgrades
No libraries were upgraded in this version.
Apache Spark
Databricks Runtime 18.1 includes Apache Spark 4.1.0. This release includes all Spark fixes and improvements included in Databricks Runtime 18.0, as well as the following additional bug fixes and improvements made to Spark:
- SPARK-55350 Fix row count loss when creating DataFrame from pandas with 0 columns
- SPARK-55364 Make SupportsIAdd and SupportsOrdering protocol more reasonable
- SPARK-53656 Refactor MemoryStream to use SparkSession instead of SQLContext
- SPARK-55472 Raise
AttributeErrorfrom methods removed in pandas 3 - SPARK-55224 Use Spark DataType as ground truth in Pandas-Arrow serialization
- SPARK-55402 Move streamingSourceIdentifyingName from CatalogTable to DataSource
- SPARK-55459 Fix 3x performance regression in applyInPandas for large groups
- SPARK-55317 Add SequentialUnion logical plan node and planning rule
- SPARK-55424 Explicitly pass the series name in
convert_numpy - SPARK-55175 Extract
to_pandastransformer from serializers - SPARK-55304 Introduce support of Admission Control and Trigger.AvailableNow in Python data source - streaming reader
- SPARK-55382 Make
Executorto logRunning Spark version - SPARK-55408 Handle unexpected keyword argument errors related to datetime with pandas 3
- SPARK-55345 Do not pass unit and closed to Timedelta for pandas 3
- SPARK-54759 Properly guard logic in Scripting Variable Manager after Cursor introduction
- SPARK-55409 Handle an unexpected keyword argument error from read_excel with pandas 3
- SPARK-55403 Fix
no attribute 'draw'error in the plot tests with pandas 3 - SPARK-55256 Revert "[SC-218596][SQL] Support IGNORE NULLS / RESPECT NULLS for array_agg and collect_list"
- SPARK-55256 Support IGNORE NULLS / RESPECT NULLS for array_agg and collect_list
- SPARK-55395 Disable RDD cache in
DataFrame.zipWithIndex - SPARK-55131 Introduce a new merge operator delimiter for RocksDB to empty string to concat without delimiter
- SPARK-55376 Make numeric_only argument in groupby functions accept only boolean with pandas 3
- SPARK-55334 Enable
TimestampTypeandTimestampNTZTypeinconvert_numpy - SPARK-55281 Add ipykernel and IPython to mypy optional package list
- SPARK-55336 Let createDF use create_batch logic for decoupling
- SPARK-55366 Remove
errorOnDuplicatedFieldNamesfrom Python UDFs - SPARK-54759 SQL scripting Cursor support
- SPARK-55302 Fix custom metrics in case of
KeyGroupedPartitioning - SPARK-55228 Implement Dataset.zipWithIndex in Scala API
- SPARK-55373 Improve noHandlerFoundForExtension error message
- SPARK-55356 Support alias for PIVOT clause
- SPARK-55359 Promote
TaskResourceRequesttoStable - SPARK-55365 Generalize the utils for arrow array conversion
- SPARK-55106 Add Repartition Integration test for TransformWithState Operators
- SPARK-55086 Add DataSourceReader.pushFilters to Python Data Source API docs
- SPARK-46165 Add support for DataFrame.all axis=None
- SPARK-55289 Revert "[SC-218749][SQL] Fix flaky test in-set-operations.sql by disabling broadcast join"
- SPARK-55297 Restore timedelta dtype based on the original dtype
- SPARK-55291 Pre-process metadata headers at client interceptor construction time
- SPARK-55155 Reapply "[SC-218401][SQL] Support foldable expressions in
SET CATALOGstatement - SPARK-55318 Performance Optimizations for vector_avg/vector_sum
- SPARK-55295 Extend the ST_GeomFromWKB function to take an optional SRID value
- SPARK-55280 Add GetStatus proto to support execution status monitoring
- SPARK-54969 Implement new arrow->pandas conversion
- SPARK-55176 Extract
arrow_to_pandasconverter into ArrowArrayToPandasConversion - SPARK-55111 Revert "[SC-217817][SC-210791][SS] Unfinished repartitioning detection on query restart"
- SPARK-55252 Improve
HttpSecurityFilterto addContent-Security-Policyheader - SPARK-55111 Unfinished repartitioning detection on query restart
- SPARK-55105 Add Integration Test for Join Operator
- SPARK-55260 Implement Parquet write support for Geo types
- SPARK-54523 Fix default resolution during variant pushdown
- SPARK-55328 Reuse PythonArrowInput.codec in GroupedPythonArrowInput
- SPARK-55246 Add Test for Pyspark TWS and TWSInPandas and Fix StatePartitionAllColumnFamiliesWriter Bug
- SPARK-55289 Fix flaky test in-set-operations.sql by disabling broadcast join
- SPARK-55040 Refactor TaskContext and related worker protocol with engine pyspark fix.
- SPARK-47996 support cross merge in pandas API
- SPARK-55031 Add vector avg/sum aggregation function expressions
- SPARK-54410 Fix read support for the variant logical type annotation
- SPARK-54776 Revert "[SC-216482][SQL] Improved the logs message regarding lambda function with SQL UDF"
- SPARK-55123 Add SequentialUnionOffset for tracking sequential source processing
- SPARK-54972 Improve NOT IN subqueries with non-nullable columns
- SPARK-54776 Improved the logs message regarding lambda function with SQL UDF
- SPARK-53807 Fix race condition issues between
unlockandreleaseAllLocksForTaskinBlockInfoManager - SPARK-51831 Revert "[SC-207389][SQL] Column pruning with existsJoin for Datasource V2"
- SPARK-54881 Improve
BooleanSimplificationto handle negation of conjunction and disjunction in one pass - SPARK-54696 Clean-up ArrowBuffers in Connect
- SPARK-55009 Remove unnecessary memory copy from the constructor of
LevelDBTypeInfo/RocksDBTypeInfo.Index - SPARK-54877 Make display stacktrace on UI error page configurable
- SPARK-51831 Column pruning with existsJoin for Datasource V2
- SPARK-55285 Fix the initialization of
PythonArrowInput - SPARK-53960 Let approx_top_k_accumulate/combine/estimate handle NULLs
- SPARK-49110 Simplify SubqueryAlias.metadataOutput to always propagate metadata columns
- SPARK-55155 Revert "[SC-218401][SQL] Support foldable expressions in
SET CATALOGstatement" - SPARK-55155 Support foldable expressions in
SET CATALOGstatement - SPARK-49110 Revert "[SC-218594][SQL] Simplify SubqueryAlias.metadataOutput to always propagate metadata columns"
- SPARK-54399 Implement the st_setsrid function in Scala and PySpark
- SPARK-49110 Simplify SubqueryAlias.metadataOutput to always propagate metadata columns
- SPARK-55133 Fix race condition in IsolatedSessionState lifecycle management
- SPARK-55262 Block Geo types in all file based data sources except Parquet
- SPARK-54202 Allow casting from GeometryType(srid) to GeometryType(ANY)
- SPARK-54142 Implement the st_srid function in Scala and PySpark
- SPARK-54887 Always set a sql state in spark connect client
- SPARK-55237 Suppress annoying messages when looking up nonexistent DBs
- SPARK-55040 Revert "[SC-217628][PYTHON] Refactor TaskContext and related worker protocol"
- SPARK-55243 Allow setting binary headers via the -bin suffix in the Scala Connect client
- SPARK-55259 Implement Parquet schema conversion for Geo types
- SPARK-55282 Avoid using worker_util in the Driver-side
- SPARK-54151 Introduce the framework for adding ST functions in PySpark
- SPARK-55194 Remove GroupArrowUDFSerializer by moving flatten logic to mapper
- SPARK-55020 Disable gc when executing gRPC command
- SPARK-55053 Refactor data source/udtf analysis workers so they have a unified entry
- SPARK-55040 Refactor TaskContext and related worker protocol
- SPARK-55244 Use np.nan as default value for pandas string types
- SPARK-55225 Restore to the original dtype for Datetime
- SPARK-55154 Remove fastpath to pd.Series for pandas 3
- SPARK-55030 Add vector norm/normalize function expressions support
- SPARK-55202 Fix UNEXPECTED_USE_OF_PARAMETER_MARKER when using param…
- SPARK-54201 Allow casting from GeographyType(srid) to GeographyType(ANY)
- SPARK-54244 Introduce type coercion support for GEOMETRY data types
- SPARK-54160 Implement the ST_SetSrid expression in SQL
- SPARK-55046 PySpark add udf processing time metric
- SPARK-54101 Introduce the framework for adding ST functions in Scala
- SPARK-54683 Unify geo and time types blocking
- SPARK-55249 Make DataFrame.toJSON able to return dataframe
- SPARK-54521 Add to and from WKB for Geometry type
- SPARK-54162 Allow casting from GeographyType to GeometryType
- SPARK-55146 Recheckin State Repartition API for PySpark
- SPARK-55140 Do not map builtin functions to numpy version for pandas 3
- SPARK-54243 Introduce type coercion support for GEOGRAPHY data types
- SPARK-55058 Throw error on inconsistent checkpoint metadata
- SPARK-55108 Use the latest pandas-stubs for type check
- SPARK-54091 Implement the ST_Srid expression in SQL
- SPARK-55104 Add Spark Connect support for DataStreamReader.name()
- SPARK-54996 Report ArrivalTime for records from LowLatencyMemoryStream
- SPARK-55118 Replace ASM Opcodes wildcard imports
- SPARK-54365 Add Repartition Integration Test for Aggregate, Dedup, FMGWS and SessionWindow Operators
- SPARK-55119 Fix Continue Handler: prevent INTERNAL_ERROR and incorrect conditional statements interruption
- SPARK-54104 Disallow casting geospatial types to/from other data types
- SPARK-55240 Refactor LazyTry stacktrace handling to use wrapper instead of suppressed exception
- SPARK-55238 Move Geo SRS mapping logic from
main/scalatomain/java - SPARK-55179 Skip eager column name validation in
df.col_name - SPARK-55055 Support SparkSession.Builder.create for PySpark Classic
- SPARK-55186 Make ArrowArrayToPandasConversion.convert_legacy able to return pd.DataFrame
- SPARK-54079 Introduce the framework for adding ST expressions in Catalyst
- SPARK-54096 Support Spatial Reference System mapping in PySpark
- SPARK-54801 Mark a few new 4.1 configs as internal
- SPARK-55146 Revert "[SC-217936][SC-210779][SS] State Repartition API for PySpark"
- SPARK-54559 Use runnerConf to pass profiler option
- SPARK-55205 Fix tests where it assumes string type will convert to object
- SPARK-55171 Fix memory profiler on iter UDF
- SPARK-55226 Recognize datetime and timedelta with units other than [ns]
- SPARK-55027 Move writeConf to PythonWorkerUtils
- SPARK-55197 Extract
_write_stream_starthelper to deduplicate START_ARROW_STREAM signal logic - SPARK-54179 Add Native Support for Apache Tuple Sketches (#190848) (#191111)
- SPARK-55151 Fix RocksDBSuite testWithStateStoreCheckpointIds
- SPARK-55146 State Repartition API for PySpark
- SPARK-54232 Enable Arrow serialization for Geography and Geometry types
- SPARK-53957 Support GEOGRAPHY and GEOMETRY in the SpatialReferenceSystemMapper
- SPARK-55169 Use ArrowBatchTransformer.flatten_struct in ArrowStreamArrowUDTFSerializer
- SPARK-55134 Fix
BasicExecutorFeatureStepto throwIllegalArgumentExceptionfor executor cpu misconfigs - SPARK-54166 Introduce type encoders for geospatial types in PySpark
- SPARK-55138 Fix
convertToMapDatathrowing aNullPointerException - SPARK-55168 Use ArrowBatchTransformer.flatten_struct in GroupArrowUDFSerializer
- SPARK-55032 Refactor profilers in workers.py
- SPARK-54990 Fix how classproperty is implemented in session.py
- SPARK-55076 Fix the type hint issue in ml/mllib and add scipy requirement
- SPARK-55162 Extract transformers from ArrowStreamUDFSerializer
- SPARK-55121 Add DataStreamReader.name() to Classic PySpark
- SPARK-54887 Add optional sqlstate to sparkthrowables
- SPARK-54169 Introduce Geography and Geometry types in Arrow writer
- SPARK-51658 Introduce Geometry and Geography in-memory wrapper formats
- SPARK-54110 Introduce type encoders for Geography and Geometry types
- SPARK-54956 Unify the indeterminate shuffle retry solution
- SPARK-55137 Refactor
GroupingAnalyticsTransformerandAnalyzercode - SPARK-54103 Introduce client-side Geography and Geometry classes
- SPARK-55160 Directly pass input schema to serializers
- SPARK-55170 Extract grouped stream reading pattern from serializers
- SPARK-55125 Remove redundant
__init__methods in Arrow serializers - SPARK-55126 Remove unused timezone and assign_cols_by_name from ArrowStreamArrowUDFSerializer
- SPARK-54980 Convert the JDBC isolation level to a string
- SPARK-55051 Byte string accepts KiB, MiB, GiB, TiB, PiB
- SPARK-55025 Improve performance in pandas by using list comprehension
- SPARK-46165 Add support for pandas.DataFrame.all axis=1
- SPARK-55037 Re-implement Observation Without Using the QueryExecutionListener
- SPARK-54965 Factor out legacy pa.Array -> pd.Series converter
- SPARK-55016 Make SQLConf a direct property of SparkSession to prevent stack overflow
- SPARK-55091 Reduce Hive RPC calls for DROP TABLE command
- SPARK-55097 Fix re-adding cached artifacts drops blocks silently issue
- SPARK-55026 Optimize BestEffortLazyVal.
- SPARK-55091 Revert "[SC-217410][SQL] Reduce Hive RPC calls for DROP TABLE command"
- SPARK-54590 Support Checkpoint V2 for State Rewriter and Repartitioning
- SPARK-55016 Make SQLConf a direct property of SparkSession to prevent stack overflow
- SPARK-55091 Reduce Hive RPC calls for DROP TABLE command
- SPARK-55016 Revert "[SC-217401][SQL] Make SQLConf a direct property of SparkSession to prevent stack overflow"
- SPARK-55016 Make SQLConf a direct property of SparkSession to prevent stack overflow
- SPARK-55098 Vectorized UDFs with output batch size control fail with memory leak
- SPARK-54824 Add support for multiGet and deleteRange for Rocksdb State Store
- SPARK-55054 Add IDENTIFIED BY support for streaming table-valued functions
- SPARK-55029 Propagate streaming source identifying name through resolution pipeline
- SPARK-55071 Make spark.addArtifact work with Windows paths
- SPARK-54103 Revert "[SC-210400][Geo][SQL] Introduce client-side Geography and Geometry classes"
- SPARK-54103 Introduce client-side Geography and Geometry classes
- SPARK-54033 Introduce Catalyst server-side geospatial execution classes
- SPARK-54176 Introduce Geography and Geometry data types to PySpark Connect
- SPARK-55089 Correct the output schema of toJSON
- SPARK-55035 Perform shuffle cleanup in child executions
- SPARK-55036 Add
ArrowTimestampConversionfor arrow timezone handling - SPARK-55090 Implement
DataFrame.toJSONin Python Client - SPARK-54873 Simplify V2TableReference resolution as only temp view may contain it
- SPARK-52828 Make hashing for collated strings collation agnostic
- SPARK-54175 Add Geography and Geometry types to Spark Connect proto
- SPARK-54961 Introduce
GroupingAnalyticsTransformer - SPARK-55088 Keep the metadata in to/from_arrow_type/schema
- SPARK-55070 Allow hidden column in dataframe column resolution
- SPARK-55044 Keep the metadata in toArrowSchema/fromArrowSchema
- SPARK-55043 Fix time travel with subquery containing table references
- SPARK-54987 Change the default value of prefer_timestamp_ntz to True in
from_arrow_type/from_arrow_schema - SPARK-54866 Refactor Drop/RefreshFunction to avoid catalog lookup
- SPARK-55024 Use REQUIRES_SINGLE_PART_NAMESPACE error for session catalog namespace validation
- SPARK-54992 Replace cast with a runtime check for make_timestamp
- SPARK-55024 Revert "[SC-216987][SQL] Use REQUIRES_SINGLE_PART_NAMESPACE error for session catalog namespace validation"
- SPARK-54866 Revert "[SC-216753][SQL] Refactor Drop/RefreshFunction to avoid catalog lookup"
- SPARK-55024 Use REQUIRES_SINGLE_PART_NAMESPACE error for session catalog namespace validation
- SPARK-54866 Refactor Drop/RefreshFunction to avoid catalog lookup
- SPARK-54991 Correct type hint for streaming/listener.py
- SPARK-54925 Add the capability to dump threads for pyspark
- SPARK-54803 Support BY NAME with INSERT ... REPLACE WHERE
- SPARK-54785 Add support for binary sketch aggregations in KLL (#188370) (#188860)
- SPARK-54949 Move pyproject.toml to repo root
- SPARK-54954 Fix remote related type hints in util.py
- SPARK-54922 Unify how args are passed to python workers
- SPARK-54870 Collation support for char/varchar and CTAS/RTAS
- SPARK-54762 Fix
_create_converterandcovertoverload signature - SPARK-55019 Allow DROP TABLE to drop VIEW
- SPARK-53103 Throw an error if state directory is not empty when query starts
- SPARK-54995 Make a fast path for foreachPartition
- SPARK-54634 Add clear error message for empty IN predicate
- SPARK-54984 State Repartition execution and integrate with State Rewriter
- SPARK-54443 Integrate PartitionKeyExtractor in Re-partition reader
- SPARK-54907 Introduce NameStreamingSources analyzer rule for streaming source evolution
- SPARK-54609 Update TIME type config to match the OSS one
- SPARK-54988 Simplify the implementation of ObservationManager.tryComplete
- SPARK-54959 Disable shuffle checksum mismatch fully retry when push based shuffle enabled
- SPARK-54940 Add tests for pa.scalar type inference
- SPARK-54634 Revert "[SC-216478][SQL] Add clear error message for empty IN predicate"
- SPARK-54337 Add support for PyCapsule to Pyspark
- SPARK-54634 Add clear error message for empty IN predicate
- SPARK-53785 Memory Source for RTM
- SPARK-54883 Clean up error messages for CLI and add new error mode DEBUG
- SPARK-54713 Add vector similarity/distance function expressions support
- SPARK-54962 Fix nullable integers handling in Pandas UDF
- SPARK-54864 Add rCTE nodes to NormalizePlan
- SPARK-53847 Add ContinuousMemorySink for Real-time Mode testing
- SPARK-54865 Add foreachWithSubqueriesAndPruning method to QueryPlan
- SPARK-54930 Remove redundant _accumulatorRegistry.clear() call in worker.py
- SPARK-54929 Fix taskContext._resources reset in loop causes only last resource to be kept
- SPARK-54963 Make
createDataFramerespectprefer_timestamp_ntzwheninfer_pandas_dict_as_map - SPARK-54920 Move grouping analytics extraction logic to a common
GroupingAnalyticsExtractor - SPARK-54924 State Rewriter to read state, transform it and write new state
- SPARK-54872 Unify column default value handling between v1 and v2 commands
- SPARK-54905 Simplify foreachWithSubqueries implementation in QueryPlan
- SPARK-54682 Support showing parameters in DESCRIBE PROCEDURE
- SPARK-54933 Avoid repeatedly fetching config
binary_as_bytesintoLocalIterator - SPARK-54872 Revert "[SC-216260][SQL] Unify column default value handling between v1 and v2 commands"
- SPARK-51936 ReplaceTableAsSelect should overwrite the new table instead of append
- SPARK-54771 Remove ResolveUserSpecifiedColumns rule from RuleIdCollection
- SPARK-54872 Unify column default value handling between v1 and v2 commands
- SPARK-54313 Add --extra-properties-file option for configuration layering
- SPARK-54468 Add missing error classes
- SPARK-46741 Cache Table with CTE should work when CTE in plan expression subquery
- SPARK-46741 Cache Table with CTE won't work
- SPARK-54615 Always pass runner_conf to python worker
- SPARK-53737 Add Real-time Mode trigger
- SPARK-54541 Rename _LEGACY_ERROR_TEMP_1007 and add sqlState
- SPARK-54718 Preserve attributes names during CTE newInstance()
- SPARK-54621 Merge Into Update Set * preserve nested fields if …coerceNestedTypes is enabled
- SPARK-54595 Keep existing behavior of MERGE INTO without SCHEMA EVOLUTION clause
- SPARK-54903 Make to_arrow_schema/to_arrow_type able to set timezone
- SPARK-52326 Add partitions related ExternalCatalogEvent and post them in corresponding operations
- SPARK-54541 Revert "[SC-215212][SQL] Rename _LEGACY_ERROR_TEMP_1007 and add sqlState"
- SPARK-54578 Perform Code Cleanup on
AssignmentUtils - SPARK-54830 Enable checksum based indeterminate shuffle retry by default
- SPARK-54525 Disable nested struct coercion in MERGE INTO under a config
- SPARK-53784 Additional Source APIs needed to support RTM execution
- SPARK-54496 Fix Merge Into Schema Evolution for Dataframe API
- SPARK-54835 Avoid unnecessary temp QueryExecution for nested command execution
- SPARK-54541 Rename _LEGACY_ERROR_TEMP_1007 and add sqlState
- SPARK-54289 Allow MERGE INTO to preserve existing struct fields for UPDATE SET * when source struct has less nested fields than target struct
- SPARK-54720 Add SparkSession.emptyDataFrame with a schema
- SPARK-54800 Changed default implementation for isObjectNotFoundException
- SPARK-54686 Relax DSv2 table checks in temp views to allow new top-level columns
- SPARK-54619 Add a sanity check for configuration numbers
- SPARK-54726 Improve a bit of performance for InsertAdaptiveSparkPlan
- SPARK-51966 Replace select.select() with select.poll() when running on POSIX os
- SPARK-54749 Fix incorrect numOutputRows metric in OneRowRelationExec
- SPARK-54411 Introduce Repartition Writer that Support Multi-CF
- SPARK-54835 Revert "[SC-215823][SQL] Avoid unnecessary temp QueryExecution for nested command execution"
- SPARK-54867 Introduce NamedStreamingRelation wrapper for source identification during analysis
- SPARK-54835 Avoid unnecessary temp QueryExecution for nested command execution
- SPARK-54491 Fix insert into temp view on DSv2 table failure
- SPARK-54871 Trim aliases from grouping and aggregate expressions before handling grouping analytics
- SPARK-51920 Fix type handling of namedTuple for transfromWithState
- SPARK-54526 Rename _LEGACY_ERROR_TEMP_1133 and add sqlState
- SPARK-54424 Failures during recaching must not fail operations
- SPARK-54894 Fix argument passing in
to_arrow_type - SPARK-53448 Conversion of a pyspark DataFrame with a Variant column to pandas fails with an error
- SPARK-54882 Remove legacy PYARROW_IGNORE_TIMEZONE
- SPARK-54504 Fix version refresh for DSv2 tables with subqueries
- SPARK-54444 Relax DSv2 table checks to restore previous behavior
- SPARK-54859 Arrow by default PySpark UDF API reference doc
- SPARK-54387 Revert "[ES-1688666] Revert "[SC-212394][SQL] Fix recaching of DSv2 tables""
- SPARK-54753 fix memory leak of ArtifactManager
- SPARK-54387 Revert "[SC-212394][SQL] Fix recaching of DSv2 tables"
- SPARK-54436 Fix error formatting for incompatible table metadata checks
- SPARK-54849 Upgrade the minimum version of
pyarrowto 18.0.0 - SPARK-54022 Make DSv2 table resolution aware of cached tables
- SPARK-54387 Fix recaching of DSv2 tables
- SPARK-53924 Reload DSv2 tables in views created using plans on each view access
- SPARK-54561 Support breakpoint() for
run-tests.py - SPARK-54157 Fix refresh of DSv2 tables in Dataset
- SPARK-54830 Revert "[CORE] Enable checksum based indeterminate shuffle retry by default"
- SPARK-54861 Reset task thread name to IDLE_TASK_THREAD_NAME when task completes
- SPARK-54834 Add new interfaces SimpleProcedure and SimpleFunction
- SPARK-54760 DelegatingCatalogExtension as session catalog supports both V1 and V2 functions
- SPARK-54685 Remove redundant observed-metrics responses
- SPARK-54853 Always check
hive.exec.max.dynamic.partitionson the spark side - SPARK-54840 OrcList Pre-allocation
- SPARK-54830 Enable checksum based indeterminate shuffle retry by default
- SPARK-54850 Improve
extractShuffleIdsto findAdaptiveSparkPlanExecanywhere in plan tree - SPARK-54843 Try_to_number expression not working for empty string input
- SPARK-54556 Rollback succeeding shuffle map stages when shuffle checksum mismatch detected
- SPARK-54760 Revert "[SC-215670][SQL] DelegatingCatalogExtension as session catalog supports both V1 and V2 functions"
- SPARK-54760 DelegatingCatalogExtension as session catalog supports both V1 and V2 functions
- SPARK-54818 TaskMemoryManager allocate failed should log errorstack to help check memory usage
- SPARK-54827 Add helper function
TreeNode.containsTag - SPARK-54777 Revert "[SC-215740][SQL] Changed dropTable error handling in JDBCTableCatalog.dropTable(...)"
- SPARK-54777 Changed dropTable error handling in JDBCTableCatalog.dropTable(...)
- SPARK-54817 Refactor
Unpivotresolution logic toUnpivotTransformer - SPARK-54820 Make
pandas_on_spark_typecompatible with numpy 2.4.0 - SPARK-54799 Refactor
UnpivotCoercion - SPARK-54754 OrcSerializer should not parse the schema every time it is serialized
- SPARK-54226 Extend Arrow compression to Pandas UDF
- SPARK-54787 Use list comprehension instead of for loops in pandas
- SPARK-54690 Make
Frame.__repr__insensitive to arrow optimization - SPARK-46166 Implementation of pandas.DataFrame.any with axis=None
- SPARK-54696 Clean-up Arrow Buffers - follow-up
- SPARK-54769 Remove dead code in conversion.py
- SPARK-54787 Use list comprehension in pandas _bool_column_labels
- SPARK-54794 Suppress verbose
FsHistoryProvider.checkForLogsscanning logs - SPARK-54745 Fix PySpark import error caused by missing UnixStreamServer on Windows
- SPARK-54782 Correct the config versions
- SPARK-54781 Return model cache information in JSON format
- SPARK-54419 Offline Repartition State Reader Supports Multi-Column Families
- SPARK-54722 Register Pandas Grouped Iter Aggregate UDF for SQL usage
- SPARK-54762 Revert "[SC-215422][PYTHON] Fix
_create_converterandcovertoverload signature" - SPARK-54652 Revert "[SC-215452][SQL] Complete conversion of IDENTIFIER()"
- SPARK-54762 Fix
_create_converterandcovertoverload signature - SPARK-52819 Making KryoSerializationCodec serializable to fix java.io.NotSerializableException errors in various use cases
- SPARK-54711 Add a timeout for daemon created worker connection
- SPARK-54738 Add Profiler Support for Pandas Grouped Iter Aggregate UDF
- SPARK-54652 Complete conversion of IDENTIFIER()
- SPARK-54581 Making fetchsize option case-insensitive for Postgres connector
- SPARK-54589 Consolidate ArrowStreamAggPandasIterUDFSerializer into ArrowStreamAggPandasUDFSerializer
- SPARK-41916 Torch distributor: support multiple torchrun processes per task if task.gpu.amount > 1
- SPARK-54707 Refactor
PIVOTresolution main logic to thePivotTransformer - SPARK-54706 Make DistributedLDAModel work with local file system
- SPARK-53616 Introduce iterator API for pandas grouped agg UDF
- SPARK-54116 Add off-heap mode support for LongHashedRelation
- SPARK-54656 Refactor SupportsPushDownVariants to be a ScanBuilder mix-in
- SPARK-54687 Add golden file with generators resolution edge cases
- SPARK-54708 Optimize ML cache cleanup with lazy directory creation
- SPARK-54116 Revert "[SC-213108][SQL] Add off-heap mode support for LongHashedRelation"
- SPARK-54116 Add off-heap mode support for LongHashedRelation
- SPARK-54443 Partition key extraction for all streaming stateful operators
- SPARK-54687 Revert "[SC-214791][SQL] Add golden file with generators resolution edge cases"
- SPARK-54116 Revert "[SC-213108][SQL] Add off-heap mode support for LongHashedRelation"
- SPARK-54687 Add golden file with generators resolution edge cases
- SPARK-54116 Add off-heap mode support for LongHashedRelation
- SPARK-54420 Introduce Offline Repartitioning StatePartitionWriter for Single Column Family
- SPARK-54689 Make
org.apache.spark.sql.pipelinesinternal package and makeEstimatorUtilsprivate - SPARK-54673 Refactor SQL pipe syntax analysis code for sharing and reuse
- SPARK-54668 Add tests for CTEs in multi-children operators
- SPARK-54669 Remove redundant casting in rCTEs
- SPARK-54587 Consolidate all runner_conf related code
- SPARK-54628 Remove all unnecessary explicit super() arguments
- SPARK-54675 Add configurable force shutdown timeout for StateStore maintenance thread pool
- SPARK-54639 Avoid unnecessary Table creation in Arrow serializers
- SPARK-49635 Remove ANSI config suggestion in CAST error messages
- SPARK-54664 Clean up the code related to
listenerCachefromconnect.StreamingQueryManager - SPARK-54640 Replace select.select with select.poll on UNIX
- SPARK-54662 Add
viztraceranddebugpyindev/requirements.txt - SPARK-54632 Add the option to use ruff for lint
- SPARK-54585 Fix State Store rollback when thread is in interrupted state
- SPARK-54172 Merge Into Schema Evolution should only add referenced columns
- SPARK-54438 Consolidate ArrowStreamAggArrowIterUDFSerializer into ArrowStreamAggArrowUDFSerializer
- SPARK-54627 Remove redundant initializations in serializers
- SPARK-54631 Add profiler support for Arrow Grouped Iter Aggregate UDF
- SPARK-54316 Reapply [SC-213971][CORE][PYTHON][SQL] Consolidate
GroupPandasIterUDFSerializerwithGroupPandasUDFSerializer - SPARK-54392 Optimize JVM-Python communication for TWS initial state
- SPARK-54617 Enable Arrow Grouped Iter Aggregate UDF registration for SQL
- SPARK-54544 Enable flake8 F811 check
- SPARK-54650 Move int to decimal conversion into
_create_converter_from_pandas - SPARK-54316 Revert "[SC-213971][CORE][PYTHON][SQL] Consolidate
GroupPandasIterUDFSerializerwithGroupPandasUDFSerializer" - SPARK-53687 Introduce WATERMARK clause in SQL statement
- SPARK-54316 Consolidate
GroupPandasIterUDFSerializerwithGroupPandasUDFSerializer - SPARK-54598 Extract logic to read UDFs
- SPARK-54622 Promote
RequiresDistributionAndOrderingand its required interfaces toEvolving - SPARK-54624 Ensure user name in historypage get escaped
- SPARK-54580 Consider Hive 4.1 in HiveVersionSuite and HiveClientImpl
- SPARK-54068 Fix
to_featherto support PyArrow 22.0.0 - SPARK-54618 Promote
LocalScantoStable - SPARK-54616 Mark
SupportsPushDownVariantsasExperimental - SPARK-54607 Remove unused method
toStringHelperfromAbstractFetchShuffleBlocks.java - SPARK-53615 Introduce iterator API for Arrow grouped aggregation UDF
- SPARK-54608 Avoid double caching of type converter in UDTF
- SPARK-54600 Don't use pickle to save/load models in
pyspark.ml.connect - SPARK-54592 Make
estimatedSizeprivate - SPARK-54388 Introduce StatePartitionReader that scan raw bytes for Single ColFamily
- SPARK-54570 Propagate the error class correctly in Spark Connect
- SPARK-54577 Optimize Py4J calls in schema inference
- SPARK-54568 Avoid unnecessary pandas conversion in create dataframe from ndarray
- SPARK-54576 Add documentation for new Datasketches-based aggregate functions
- SPARK-54574 Reenable FPGrowth on connect
- SPARK-54557 Make CSV/JSON/XmlOptions and CSV/JSON/XmlInferSchema comparable
- SPARK-52798 Add function approx_top_k_combine
- SPARK-54446 FPGrowth supports local filesystem with Arrow file format
- SPARK-54547 Rename
hostPortvariable tohostinTaskSchedulerImpl.(executorLost|logExecutorLoss)methods - SPARK-54558 Fix Internal Exception when Exception Handlers with no BEGIN/END are used
- SPARK-52923 Allow ShuffleManager to control push merge during shuffle registration
- SPARK-54474 Discard the XML report on tests that are supposed to fail
- SPARK-54473 Add Avro read and write support for TIME type
- SPARK-54472 Add ORC read and write support for TIME type
- SPARK-54463 Add CSV serialization and deserialization support for TIME type
- SPARK-52588 Approx_top_k: accumulate and estimate
- SPARK-54461 Add XML serialization and deserialization support for TIME type
- SPARK-54451 Add JSON serialization and deserialization support for TIME type
- SPARK-54537 Fix SparkConnectDatabaseMetaData getSchemas/getTables on catalogs missing namespace capbalitity
- SPARK-54442 Add numeric conversion functions for TIME type
- SPARK-54451 Revert "[SC-212861][SQL] Add JSON serialization and deserialization support for TIME type"
- SPARK-54492 Rename _LEGACY_ERROR_TEMP_1201 and add sqlState
- SPARK-54531 Introduce ArrowStreamAggPandasUDFSerializer
- SPARK-54223 Add task context and data metrics to Python runner logs
- SPARK-54272 Add aggTime for SortAggregateExec
- SPARK-53469 Ability to cleanup shuffle in Thrift server
- SPARK-54219 Support
spark.cleaner.referenceTracking.blocking.timeoutconfig - SPARK-54475 Add master-server, branch-4.0-client, Python 3.11
- SPARK-54451 Add JSON serialization and deserialization support for TIME type
- SPARK-54285 Cache timezone info to avoid expensive timestamp conversion
- SPARK-49133 Make member
MemoryConsumer#usedatomic to avoid user code causing deadlock - SPARK-46166 Implementation of pandas.DataFrame.any with axis=1
- SPARK-54532 Add support for sqlstate for PySparkException
- SPARK-54435 spark-pipelines init should avoid overwriting existing directory
- SPARK-54247 Explictly close socket for util._load_from_socket
Databricks ODBC/JDBC driver support
Databricks supports ODBC/JDBC drivers released in the past 2 years. Please download the recently released drivers and upgrade (download ODBC, download JDBC).
System environment
- Operating System: Ubuntu 24.04.3 LTS
- Java: Zulu21.42+19-CA
- Scala: 2.13.16
- Python: 3.12.3
- R: 4.5.1
- Delta Lake: 4.0.1
Installed Python libraries
| Library | Version | Library | Version | Library | Version |
|---|---|---|---|---|---|
| aiohappyeyeballs | 2.4.4 | aiohttp | 3.11.10 | aiosignal | 1.2.0 |
| annotated-doc | 0.0.4 | annotated-types | 0.7.0 | anyio | 4.7.0 |
| argon2-cffi | 21.3.0 | argon2-cffi-bindings | 21.2.0 | arro3-core | 0.6.5 |
| arrow | 1.3.0 | asttokens | 3.0.0 | astunparse | 1.6.3 |
| async-lru | 2.0.4 | attrs | 24.3.0 | autocommand | 2.2.2 |
| azure-common | 1.1.28 | azure-core | 1.37.0 | azure-identity | 1.20.0 |
| azure-mgmt-core | 1.6.0 | azure-mgmt-web | 8.0.0 | azure-storage-blob | 12.28.0 |
| azure-storage-file-datalake | 12.22.0 | babel | 2.16.0 | backports.tarfile | 1.2.0 |
| beautifulsoup4 | 4.12.3 | black | 24.10.0 | bleach | 6.2.0 |
| blinker | 1.7.0 | boto3 | 1.40.45 | botocore | 1.40.45 |
| cachetools | 5.5.1 | certifi | 2025.4.26 | cffi | 1.17.1 |
| chardet | 4.0.0 | charset-normalizer | 3.3.2 | click | 8.1.8 |
| cloudpickle | 3.0.0 | comm | 0.2.1 | contourpy | 1.3.1 |
| cryptography | 44.0.1 | cycler | 0.11.0 | Cython | 3.1.5 |
| databricks-agents | 1.9.1 | databricks-sdk | 0.67.0 | dataclasses-json | 0.6.7 |
| dbus-python | 1.3.2 | debugpy | 1.8.11 | decorator | 5.1.1 |
| defusedxml | 0.7.1 | deltalake | 1.1.4 | Deprecated | 1.2.18 |
| distlib | 0.3.9 | docstring-to-markdown | 0.11 | executing | 1.2.0 |
| facets-overview | 1.1.1 | fastapi | 0.128.0 | fastjsonschema | 2.21.1 |
| filelock | 3.17.0 | fonttools | 4.55.3 | fqdn | 1.5.1 |
| frozenlist | 1.5.0 | fsspec | 2023.5.0 | gitdb | 4.0.11 |
| GitPython | 3.1.43 | google-api-core | 2.28.1 | google-auth | 2.47.0 |
| google-cloud-core | 2.5.0 | google-cloud-storage | 3.7.0 | google-crc32c | 1.8.0 |
| google-resumable-media | 2.8.0 | googleapis-common-protos | 1.65.0 | grpcio | 1.67.0 |
| grpcio-status | 1.67.0 | h11 | 0.16.0 | hf-xet | 1.2.0 |
| httpcore | 1.0.9 | httplib2 | 0.20.4 | httpx | 0.28.1 |
| huggingface_hub | 1.2.4 | idna | 3.7 | importlib_metadata | 8.5.0 |
| inflect | 7.3.1 | iniconfig | 1.1.1 | ipyflow-core | 0.0.209 |
| ipykernel | 6.29.5 | ipython | 8.30.0 | ipython-genutils | 0.2.0 |
| ipywidgets | 7.8.1 | isodate | 0.7.2 | isoduration | 20.11.0 |
| jaraco.collections | 5.1.0 | jaraco.context | 5.3.0 | jaraco.functools | 4.0.1 |
| jaraco.text | 3.12.1 | jedi | 0.19.2 | Jinja2 | 3.1.6 |
| jiter | 0.12.0 | jmespath | 1.0.1 | joblib | 1.4.2 |
| json5 | 0.9.25 | jsonpatch | 1.33 | jsonpointer | 3.0.0 |
| jsonschema | 4.23.0 | jsonschema-specifications | 2023.7.1 | jupyter-events | 0.12.0 |
| jupyter-lsp | 2.2.5 | jupyter_client | 8.6.3 | jupyter_core | 5.7.2 |
| jupyter_server | 2.15.0 | jupyter_server_terminals | 0.5.3 | jupyterlab | 4.3.4 |
| jupyterlab_pygments | 0.3.0 | jupyterlab_server | 2.27.3 | jupyterlab_widgets | 1.1.11 |
| kiwisolver | 1.4.8 | langchain-core | 1.2.6 | langchain-openai | 1.1.6 |
| langsmith | 0.6.1 | launchpadlib | 1.11.0 | lazr.restfulclient | 0.14.6 |
| lazr.uri | 1.0.6 | litellm | 1.75.9 | markdown-it-py | 2.2.0 |
| MarkupSafe | 3.0.2 | marshmallow | 3.26.2 | matplotlib | 3.10.0 |
| matplotlib-inline | 0.1.7 | mccabe | 0.7.0 | mdurl | 0.1.0 |
| mistune | 3.1.2 | mlflow-skinny | 3.8.1 | mmh3 | 5.2.0 |
| more-itertools | 10.3.0 | msal | 1.34.0 | msal-extensions | 1.3.1 |
| multidict | 6.1.0 | mypy-extensions | 1.0.0 | nbclient | 0.10.2 |
| nbconvert | 7.16.6 | nbformat | 5.10.4 | nest-asyncio | 1.6.0 |
| nodeenv | 1.10.0 | notebook | 7.3.2 | notebook_shim | 0.2.4 |
| numpy | 2.1.3 | oauthlib | 3.2.2 | openai | 2.14.0 |
| opentelemetry-api | 1.39.1 | opentelemetry-proto | 1.39.1 | opentelemetry-sdk | 1.39.1 |
| opentelemetry-semantic-conventions | 0.60b1 | orjson | 3.11.5 | overrides | 7.4.0 |
| packaging | 24.2 | pandas | 2.2.3 | pandocfilters | 1.5.0 |
| parso | 0.8.4 | pathspec | 0.10.3 | patsy | 1.0.1 |
| pexpect | 4.8.0 | pillow | 11.1.0 | pip | 25.0.1 |
| platformdirs | 4.3.7 | plotly | 5.24.1 | pluggy | 1.5.0 |
| prometheus_client | 0.21.1 | prompt-toolkit | 3.0.43 | propcache | 0.3.1 |
| proto-plus | 1.27.0 | protobuf | 5.29.4 | psutil | 5.9.0 |
| psycopg2 | 2.9.11 | ptyprocess | 0.7.0 | pure-eval | 0.2.2 |
| pyarrow | 21.0.0 | pyasn1 | 0.4.8 | pyasn1-modules | 0.2.8 |
| pyccolo | 0.0.71 | pycparser | 2.21 | pydantic | 2.10.6 |
| pydantic_core | 2.27.2 | pyflakes | 3.2.0 | Pygments | 2.19.1 |
| PyGObject | 3.48.2 | pyiceberg | 0.10.0 | PyJWT | 2.10.1 |
| pyodbc | 5.2.0 | pyparsing | 3.2.0 | pyright | 1.1.394 |
| pyroaring | 1.0.3 | pytest | 8.3.5 | python-dateutil | 2.9.0.post0 |
| python-dotenv | 1.2.1 | python-json-logger | 3.2.1 | python-lsp-jsonrpc | 1.1.2 |
| python-lsp-server | 1.12.2 | pytoolconfig | 1.2.6 | pytz | 2024.1 |
| PyYAML | 6.0.2 | pyzmq | 26.2.0 | referencing | 0.30.2 |
| regex | 2024.11.6 | requests | 2.32.3 | requests-toolbelt | 1.0.0 |
| rfc3339-validator | 0.1.4 | rfc3986-validator | 0.1.1 | rich | 13.9.4 |
| rope | 1.13.0 | rpds-py | 0.22.3 | rsa | 4.9.1 |
| s3transfer | 0.14.0 | scikit-learn | 1.6.1 | scipy | 1.15.3 |
| seaborn | 0.13.2 | Send2Trash | 1.8.2 | setuptools | 78.1.1 |
| shellingham | 1.5.4 | six | 1.17.0 | smmap | 5.0.0 |
| sniffio | 1.3.0 | sortedcontainers | 2.4.0 | soupsieve | 2.5 |
| sqlparse | 0.5.5 | ssh-import-id | 5.11 | stack-data | 0.6.3 |
| starlette | 0.50.0 | strictyaml | 1.7.3 | tenacity | 9.0.0 |
| terminado | 0.17.1 | threadpoolctl | 3.5.0 | tiktoken | 0.12.0 |
| tinycss2 | 1.4.0 | tokenize_rt | 6.1.0 | tokenizers | 0.22.2 |
| tomli | 2.0.1 | tornado | 6.5.1 | tqdm | 4.67.1 |
| traitlets | 5.14.3 | typeguard | 4.3.0 | typer-slim | 0.21.1 |
| types-python-dateutil | 2.9.0.20251115 | typing-inspect | 0.9.0 | typing_extensions | 4.12.2 |
| tzdata | 2024.1 | ujson | 5.10.0 | unattended-upgrades | 0.1 |
| uri-template | 1.3.0 | urllib3 | 2.3.0 | uuid_utils | 0.12.0 |
| uvicorn | 0.40.0 | virtualenv | 20.29.3 | wadllib | 1.3.6 |
| wcwidth | 0.2.5 | webcolors | 25.10.0 | webencodings | 0.5.1 |
| websocket-client | 1.8.0 | whatthepatch | 1.0.2 | wheel | 0.45.1 |
| whenever | 0.7.3 | widgetsnbextension | 3.6.6 | wrapt | 1.17.0 |
| yapf | 0.40.2 | yarl | 1.18.0 | zipp | 3.21.0 |
| zstandard | 0.23.0 |
Installed R libraries
R libraries are installed from the Posit Package Manager CRAN snapshot on 2025-11-20.
| Library | Version | Library | Version | Library | Version |
|---|---|---|---|---|---|
| arrow | 22.0.0 | askpass | 1.2.1 | assertthat | 0.2.1 |
| backports | 1.5.0 | base | 4.5.1 | base64enc | 0.1-3 |
| bigD | 0.3.1 | bit | 4.6.0 | bit64 | 4.6.0-1 |
| bitops | 1.0-9 | blob | 1.2.4 | boot | 1.3-30 |
| brew | 1.0-10 | brio | 1.1.5 | broom | 1.0.10 |
| bslib | 0.9.0 | cachem | 1.1.0 | callr | 3.7.6 |
| caret | 7.0-1 | cellranger | 1.1.0 | chron | 2.3-62 |
| class | 7.3-22 | cli | 3.6.5 | clipr | 0.8.0 |
| clock | 0.7.3 | cluster | 2.1.6 | codetools | 0.2-20 |
| commonmark | 2.0.0 | compiler | 4.5.1 | config | 0.3.2 |
| conflicted | 1.2.0 | cpp11 | 0.5.2 | crayon | 1.5.3 |
| credentials | 2.0.3 | curl | 7.0.0 | data.table | 1.17.8 |
| datasets | 4.5.1 | DBI | 1.2.3 | dbplyr | 2.5.1 |
| desc | 1.4.3 | devtools | 2.4.6 | diagram | 1.6.5 |
| diffobj | 0.3.6 | digest | 0.6.39 | downlit | 0.4.5 |
| dplyr | 1.1.4 | dtplyr | 1.3.2 | e1071 | 1.7-16 |
| ellipsis | 0.3.2 | evaluate | 1.0.5 | fansi | 1.0.7 |
| farver | 2.1.2 | fastmap | 1.2.0 | fontawesome | 0.5.3 |
| forcats | 1.0.1 | foreach | 1.5.2 | foreign | 0.8-86 |
| fs | 1.6.6 | future | 1.68.0 | future.apply | 1.20.0 |
| gargle | 1.6.0 | generics | 0.1.4 | gert | 2.2.0 |
| ggplot2 | 4.0.1 | gh | 1.5.0 | git2r | 0.36.2 |
| gitcreds | 0.1.2 | glmnet | 4.1-10 | globals | 0.18.0 |
| glue | 1.8.0 | googledrive | 2.1.2 | googlesheets4 | 1.1.2 |
| gower | 1.0.2 | graphics | 4.5.1 | grDevices | 4.5.1 |
| grid | 4.5.1 | gridExtra | 2.3 | gsubfn | 0.7 |
| gt | 1.1.0 | gtable | 0.3.6 | hardhat | 1.4.2 |
| haven | 2.5.5 | highr | 0.11 | hms | 1.1.4 |
| htmltools | 0.5.8.1 | htmlwidgets | 1.6.4 | httpuv | 1.6.16 |
| httr | 1.4.7 | httr2 | 1.2.1 | ids | 1.0.1 |
| ini | 0.3.1 | ipred | 0.9-15 | isoband | 0.2.7 |
| iterators | 1.0.14 | jquerylib | 0.1.4 | jsonlite | 2.0.0 |
| juicyjuice | 0.1.0 | KernSmooth | 2.23-22 | knitr | 1.50 |
| labeling | 0.4.3 | later | 1.4.4 | lattice | 0.22-5 |
| lava | 1.8.2 | lifecycle | 1.0.4 | listenv | 0.10.0 |
| litedown | 0.8 | lubridate | 1.9.4 | magrittr | 2.0.4 |
| markdown | 2.0 | MASS | 7.3-60.0.1 | Matrix | 1.6-5 |
| memoise | 2.0.1 | methods | 4.5.1 | mgcv | 1.9-1 |
| mime | 0.13 | miniUI | 0.1.2 | mlflow | 3.6.0 |
| ModelMetrics | 1.2.2.2 | modelr | 0.1.11 | nlme | 3.1-164 |
| nnet | 7.3-19 | numDeriv | 2016.8-1.1 | openssl | 2.3.4 |
| otel | 0.2.0 | parallel | 4.5.1 | parallelly | 1.45.1 |
| pillar | 1.11.1 | pkgbuild | 1.4.8 | pkgconfig | 2.0.3 |
| pkgdown | 2.2.0 | pkgload | 1.4.1 | plogr | 0.2.0 |
| plyr | 1.8.9 | praise | 1.0.0 | prettyunits | 1.2.0 |
| pROC | 1.19.0.1 | processx | 3.8.6 | prodlim | 2025.04.28 |
| profvis | 0.4.0 | progress | 1.2.3 | progressr | 0.18.0 |
| promises | 1.5.0 | proto | 1.0.0 | proxy | 0.4-27 |
| ps | 1.9.1 | purrr | 1.2.0 | R6 | 2.6.1 |
| ragg | 1.5.0 | randomForest | 4.7-1.2 | rappdirs | 0.3.3 |
| rcmdcheck | 1.4.0 | RColorBrewer | 1.1-3 | Rcpp | 1.1.0 |
| RcppEigen | 0.3.4.0.2 | reactable | 0.4.4 | reactR | 0.6.1 |
| readr | 2.1.6 | readxl | 1.4.5 | recipes | 1.3.1 |
| rematch | 2.0.0 | rematch2 | 2.1.2 | remotes | 2.5.0 |
| reprex | 2.1.1 | reshape2 | 1.4.5 | rlang | 1.1.6 |
| rmarkdown | 2.30 | RODBC | 1.3-26 | roxygen2 | 7.3.3 |
| rpart | 4.1.23 | rprojroot | 2.1.1 | Rserve | 1.8-15 |
| RSQLite | 2.4.4 | rstudioapi | 0.17.1 | rversions | 3.0.0 |
| rvest | 1.0.5 | S7 | 0.2.1 | sass | 0.4.10 |
| scales | 1.4.0 | selectr | 0.4-2 | sessioninfo | 1.2.3 |
| shape | 1.4.6.1 | shiny | 1.11.1 | sourcetools | 0.1.7-1 |
| sparklyr | 1.9.3 | SparkR | 4.1.0 | sparsevctrs | 0.3.4 |
| spatial | 7.3-17 | splines | 4.5.1 | sqldf | 0.4-11 |
| SQUAREM | 2021.1 | stats | 4.5.1 | stats4 | 4.5.1 |
| stringi | 1.8.7 | stringr | 1.6.0 | survival | 3.5-8 |
| swagger | 5.17.14.1 | sys | 3.4.3 | systemfonts | 1.3.1 |
| tcltk | 4.5.1 | testthat | 3.3.0 | textshaping | 1.0.4 |
| tibble | 3.3.0 | tidyr | 1.3.1 | tidyselect | 1.2.1 |
| tidyverse | 2.0.0 | timechange | 0.3.0 | timeDate | 4051.111 |
| tinytex | 0.58 | tools | 4.5.1 | tzdb | 0.5.0 |
| urlchecker | 1.0.1 | usethis | 3.2.1 | utf8 | 1.2.6 |
| utils | 4.5.1 | uuid | 1.2-1 | V8 | 8.0.1 |
| vctrs | 0.6.5 | viridisLite | 0.4.2 | vroom | 1.6.6 |
| waldo | 0.6.2 | whisker | 0.4.1 | withr | 3.0.2 |
| xfun | 0.54 | xml2 | 1.5.0 | xopen | 1.0.1 |
| xtable | 1.8-4 | yaml | 2.3.10 | zeallot | 0.2.0 |
| zip | 2.3.3 |
Installed Java and Scala libraries (Scala 2.13 cluster version)
| Group ID | Artifact ID | Version |
|---|---|---|
| antlr | antlr | 2.7.7 |
| com.amazonaws | amazon-kinesis-client | 1.15.3 |
| com.amazonaws | aws-java-sdk-autoscaling | 1.12.681 |
| com.amazonaws | aws-java-sdk-cloudformation | 1.12.681 |
| com.amazonaws | aws-java-sdk-cloudfront | 1.12.681 |
| com.amazonaws | aws-java-sdk-cloudhsm | 1.12.681 |
| com.amazonaws | aws-java-sdk-cloudsearch | 1.12.681 |
| com.amazonaws | aws-java-sdk-cloudtrail | 1.12.681 |
| com.amazonaws | aws-java-sdk-cloudwatch | 1.12.681 |
| com.amazonaws | aws-java-sdk-cloudwatchmetrics | 1.12.681 |
| com.amazonaws | aws-java-sdk-codedeploy | 1.12.681 |
| com.amazonaws | aws-java-sdk-cognitoidentity | 1.12.681 |
| com.amazonaws | aws-java-sdk-cognitosync | 1.12.681 |
| com.amazonaws | aws-java-sdk-config | 1.12.681 |
| com.amazonaws | aws-java-sdk-core | 1.12.681 |
| com.amazonaws | aws-java-sdk-datapipeline | 1.12.681 |
| com.amazonaws | aws-java-sdk-directconnect | 1.12.681 |
| com.amazonaws | aws-java-sdk-directory | 1.12.681 |
| com.amazonaws | aws-java-sdk-dynamodb | 1.12.681 |
| com.amazonaws | aws-java-sdk-ec2 | 1.12.681 |
| com.amazonaws | aws-java-sdk-ecs | 1.12.681 |
| com.amazonaws | aws-java-sdk-efs | 1.12.681 |
| com.amazonaws | aws-java-sdk-elasticache | 1.12.681 |
| com.amazonaws | aws-java-sdk-elasticbeanstalk | 1.12.681 |
| com.amazonaws | aws-java-sdk-elasticloadbalancing | 1.12.681 |
| com.amazonaws | aws-java-sdk-elastictranscoder | 1.12.681 |
| com.amazonaws | aws-java-sdk-emr | 1.12.681 |
| com.amazonaws | aws-java-sdk-glacier | 1.12.681 |
| com.amazonaws | aws-java-sdk-glue | 1.12.681 |
| com.amazonaws | aws-java-sdk-iam | 1.12.681 |
| com.amazonaws | aws-java-sdk-importexport | 1.12.681 |
| com.amazonaws | aws-java-sdk-kinesis | 1.12.681 |
| com.amazonaws | aws-java-sdk-kms | 1.12.681 |
| com.amazonaws | aws-java-sdk-lambda | 1.12.681 |
| com.amazonaws | aws-java-sdk-logs | 1.12.681 |
| com.amazonaws | aws-java-sdk-machinelearning | 1.12.681 |
| com.amazonaws | aws-java-sdk-opsworks | 1.12.681 |
| com.amazonaws | aws-java-sdk-rds | 1.12.681 |
| com.amazonaws | aws-java-sdk-redshift | 1.12.681 |
| com.amazonaws | aws-java-sdk-route53 | 1.12.681 |
| com.amazonaws | aws-java-sdk-s3 | 1.12.681 |
| com.amazonaws | aws-java-sdk-ses | 1.12.681 |
| com.amazonaws | aws-java-sdk-simpledb | 1.12.681 |
| com.amazonaws | aws-java-sdk-simpleworkflow | 1.12.681 |
| com.amazonaws | aws-java-sdk-sns | 1.12.681 |
| com.amazonaws | aws-java-sdk-sqs | 1.12.681 |
| com.amazonaws | aws-java-sdk-ssm | 1.12.681 |
| com.amazonaws | aws-java-sdk-storagegateway | 1.12.681 |
| com.amazonaws | aws-java-sdk-sts | 1.12.681 |
| com.amazonaws | aws-java-sdk-support | 1.12.681 |
| com.amazonaws | aws-java-sdk-workspaces | 1.12.681 |
| com.amazonaws | jmespath-java | 1.12.681 |
| com.clearspring.analytics | stream | 2.9.8 |
| com.databricks | Rserve | 1.8-3 |
| com.databricks | databricks-sdk-java | 0.53.0 |
| com.databricks | jets3t | 0.7.1-0 |
| com.databricks.scalapb | scalapb-runtime_2.13 | 0.4.15-11 |
| com.esotericsoftware | kryo-shaded | 4.0.3 |
| com.esotericsoftware | minlog | 1.3.0 |
| com.fasterxml | classmate | 1.5.1 |
| com.fasterxml.jackson.core | jackson-annotations | 2.18.2 |
| com.fasterxml.jackson.core | jackson-core | 2.18.2 |
| com.fasterxml.jackson.core | jackson-databind | 2.18.2 |
| com.fasterxml.jackson.dataformat | jackson-dataformat-cbor | 2.18.2 |
| com.fasterxml.jackson.dataformat | jackson-dataformat-yaml | 2.15.2 |
| com.fasterxml.jackson.datatype | jackson-datatype-joda | 2.18.2 |
| com.fasterxml.jackson.datatype | jackson-datatype-jsr310 | 2.18.3 |
| com.fasterxml.jackson.module | jackson-module-paranamer | 2.18.2 |
| com.fasterxml.jackson.module | jackson-module-scala_2.13 | 2.18.2 |
| com.github.ben-manes.caffeine | caffeine | 2.9.3 |
| com.github.blemale | scaffeine_2.13 | 4.1.0 |
| com.github.fommil | jniloader | 1.1 |
| com.github.fommil.netlib | native_ref-java | 1.1 |
| com.github.fommil.netlib | native_ref-java | 1.1-natives |
| com.github.fommil.netlib | native_system-java | 1.1 |
| com.github.fommil.netlib | native_system-java | 1.1-natives |
| com.github.fommil.netlib | netlib-native_ref-linux-x86_64 | 1.1-natives |
| com.github.fommil.netlib | netlib-native_system-linux-x86_64 | 1.1-natives |
| com.github.luben | zstd-jni | 1.5.7-6 |
| com.github.virtuald | curvesapi | 1.08 |
| com.github.wendykierp | JTransforms | 3.1 |
| com.google.api.grpc | proto-google-common-protos | 2.5.1 |
| com.google.auth | google-auth-library-credentials | 1.20.0 |
| com.google.auth | google-auth-library-oauth2-http | 1.20.0 |
| com.google.auto.value | auto-value-annotations | 1.10.4 |
| com.google.code.findbugs | jsr305 | 3.0.0 |
| com.google.code.gson | gson | 2.11.0 |
| com.google.crypto.tink | tink | 1.16.0 |
| com.google.errorprone | error_prone_annotations | 2.36.0 |
| com.google.flatbuffers | flatbuffers-java | 25.2.10 |
| com.google.guava | failureaccess | 1.0.3 |
| com.google.guava | guava | 33.4.8-jre |
| com.google.http-client | google-http-client | 1.43.3 |
| com.google.http-client | google-http-client-gson | 1.43.3 |
| com.google.j2objc | j2objc-annotations | 3.0.0 |
| com.google.protobuf | protobuf-java | 3.25.5 |
| com.google.protobuf | protobuf-java-util | 3.25.5 |
| com.helger | profiler | 1.1.1 |
| com.ibm.icu | icu4j | 75.1 |
| com.jcraft | jsch | 0.1.55 |
| com.lihaoyi | sourcecode_2.13 | 0.1.9 |
| com.microsoft.azure | azure-data-lake-store-sdk | 2.3.10 |
| com.microsoft.sqlserver | mssql-jdbc | 12.8.0.jre11 |
| com.microsoft.sqlserver | mssql-jdbc | 12.8.0.jre8 |
| com.ning | compress-lzf | 1.1.2 |
| com.sun.mail | javax.mail | 1.5.2 |
| com.sun.xml.bind | jaxb-core | 2.2.11 |
| com.sun.xml.bind | jaxb-impl | 2.2.11 |
| com.tdunning | json | 1.8 |
| com.thoughtworks.paranamer | paranamer | 2.8 |
| com.trueaccord.lenses | lenses_2.13 | 0.4.13 |
| com.twitter | chill-java | 0.10.0 |
| com.twitter | chill_2.13 | 0.10.0 |
| com.twitter | util-app_2.13 | 19.8.1 |
| com.twitter | util-core_2.13 | 19.8.1 |
| com.twitter | util-function_2.13 | 19.8.1 |
| com.twitter | util-jvm_2.13 | 19.8.1 |
| com.twitter | util-lint_2.13 | 19.8.1 |
| com.twitter | util-registry_2.13 | 19.8.1 |
| com.twitter | util-stats_2.13 | 19.8.1 |
| com.typesafe | config | 1.4.3 |
| com.typesafe.scala-logging | scala-logging_2.13 | 3.9.2 |
| com.uber | h3 | 3.7.3 |
| com.univocity | univocity-parsers | 2.9.1 |
| com.zaxxer | HikariCP | 4.0.3 |
| com.zaxxer | SparseBitSet | 1.3 |
| commons-cli | commons-cli | 1.10.0 |
| commons-codec | commons-codec | 1.19.0 |
| commons-collections | commons-collections | 3.2.2 |
| commons-dbcp | commons-dbcp | 1.4 |
| commons-fileupload | commons-fileupload | 1.6.0 |
| commons-httpclient | commons-httpclient | 3.1 |
| commons-io | commons-io | 2.21.0 |
| commons-lang | commons-lang | 2.6 |
| commons-logging | commons-logging | 1.1.3 |
| commons-pool | commons-pool | 1.5.4 |
| dev.ludovic.netlib | arpack | 3.0.4 |
| dev.ludovic.netlib | blas | 3.0.4 |
| dev.ludovic.netlib | lapack | 3.0.4 |
| info.ganglia.gmetric4j | gmetric4j | 1.0.10 |
| io.airlift | aircompressor | 2.0.2 |
| io.delta | delta-sharing-client_2.13 | 1.3.9 |
| io.dropwizard.metrics | metrics-annotation | 4.2.37 |
| io.dropwizard.metrics | metrics-core | 4.2.37 |
| io.dropwizard.metrics | metrics-graphite | 4.2.37 |
| io.dropwizard.metrics | metrics-healthchecks | 4.2.37 |
| io.dropwizard.metrics | metrics-jetty10 | 4.2.37 |
| io.dropwizard.metrics | metrics-jmx | 4.2.37 |
| io.dropwizard.metrics | metrics-json | 4.2.37 |
| io.dropwizard.metrics | metrics-jvm | 4.2.37 |
| io.dropwizard.metrics | metrics-servlets | 4.2.37 |
| io.github.java-diff-utils | java-diff-utils | 4.15 |
| io.netty | netty-all | 4.2.7.Final |
| io.netty | netty-buffer | 4.2.7.Final |
| io.netty | netty-codec | 4.2.7.Final |
| io.netty | netty-codec-base | 4.2.7.Final |
| io.netty | netty-codec-classes-quic | 4.2.7.Final |
| io.netty | netty-codec-compression | 4.2.7.Final |
| io.netty | netty-codec-http | 4.2.7.Final |
| io.netty | netty-codec-http2 | 4.2.7.Final |
| io.netty | netty-codec-http3 | 4.2.7.Final |
| io.netty | netty-codec-marshalling | 4.2.7.Final |
| io.netty | netty-codec-native-quic | 4.2.7.Final-linux-aarch_64 |
| io.netty | netty-codec-native-quic | 4.2.7.Final-linux-x86_64 |
| io.netty | netty-codec-native-quic | 4.2.7.Final-osx-aarch_64 |
| io.netty | netty-codec-native-quic | 4.2.7.Final-osx-x86_64 |
| io.netty | netty-codec-native-quic | 4.2.7.Final-windows-x86_64 |
| io.netty | netty-codec-protobuf | 4.2.7.Final |
| io.netty | netty-codec-socks | 4.2.7.Final |
| io.netty | netty-common | 4.2.7.Final |
| io.netty | netty-handler | 4.2.7.Final |
| io.netty | netty-handler-proxy | 4.2.7.Final |
| io.netty | netty-resolver | 4.2.7.Final |
| io.netty | netty-tcnative-boringssl-static | 2.0.74.Final-db-r0-linux-aarch_64 |
| io.netty | netty-tcnative-boringssl-static | 2.0.74.Final-db-r0-linux-x86_64 |
| io.netty | netty-tcnative-boringssl-static | 2.0.74.Final-db-r0-osx-aarch_64 |
| io.netty | netty-tcnative-boringssl-static | 2.0.74.Final-db-r0-osx-x86_64 |
| io.netty | netty-tcnative-boringssl-static | 2.0.74.Final-db-r0-windows-x86_64 |
| io.netty | netty-tcnative-classes | 2.0.74.Final |
| io.netty | netty-transport | 4.2.7.Final |
| io.netty | netty-transport-classes-epoll | 4.2.7.Final |
| io.netty | netty-transport-classes-io_uring | 4.2.7.Final |
| io.netty | netty-transport-classes-kqueue | 4.2.7.Final |
| io.netty | netty-transport-native-epoll | 4.2.7.Final |
| io.netty | netty-transport-native-epoll | 4.2.7.Final-linux-aarch_64 |
| io.netty | netty-transport-native-epoll | 4.2.7.Final-linux-riscv64 |
| io.netty | netty-transport-native-epoll | 4.2.7.Final-linux-x86_64 |
| io.netty | netty-transport-native-io_uring | 4.2.7.Final-linux-aarch_64 |
| io.netty | netty-transport-native-io_uring | 4.2.7.Final-linux-riscv64 |
| io.netty | netty-transport-native-io_uring | 4.2.7.Final-linux-x86_64 |
| io.netty | netty-transport-native-kqueue | 4.2.7.Final-osx-aarch_64 |
| io.netty | netty-transport-native-kqueue | 4.2.7.Final-osx-x86_64 |
| io.netty | netty-transport-native-unix-common | 4.2.7.Final |
| io.opencensus | opencensus-api | 0.31.1 |
| io.opencensus | opencensus-contrib-http-util | 0.31.1 |
| io.prometheus | simpleclient | 0.16.1-databricks |
| io.prometheus | simpleclient_common | 0.16.1-databricks |
| io.prometheus | simpleclient_dropwizard | 0.16.1-databricks |
| io.prometheus | simpleclient_pushgateway | 0.16.1-databricks |
| io.prometheus | simpleclient_servlet | 0.16.1-databricks |
| io.prometheus | simpleclient_servlet_common | 0.16.1-databricks |
| io.prometheus | simpleclient_tracer_common | 0.16.1-databricks |
| io.prometheus | simpleclient_tracer_otel | 0.16.1-databricks |
| io.prometheus | simpleclient_tracer_otel_agent | 0.16.1-databricks |
| io.prometheus.jmx | collector | 0.18.0 |
| jakarta.annotation | jakarta.annotation-api | 1.3.5 |
| jakarta.servlet | jakarta.servlet-api | 4.0.3 |
| jakarta.validation | jakarta.validation-api | 2.0.2 |
| jakarta.ws.rs | jakarta.ws.rs-api | 2.1.6 |
| javax.activation | activation | 1.1.1 |
| javax.annotation | javax.annotation-api | 1.3.2 |
| javax.el | javax.el-api | 2.2.4 |
| javax.jdo | jdo-api | 3.0.1 |
| javax.media | jai_core | jai_core_dummy |
| javax.transaction | jta | 1.1 |
| javax.transaction | transaction-api | 1.1 |
| javax.xml.bind | jaxb-api | 2.2.11 |
| javolution | javolution | 5.5.1 |
| jline | jline | 2.14.6 |
| joda-time | joda-time | 2.14.0 |
| net.java.dev.jna | jna | 5.8.0 |
| net.razorvine | pickle | 1.5 |
| net.sf.jpam | jpam | 1.1 |
| net.sf.opencsv | opencsv | 2.3 |
| net.sf.supercsv | super-csv | 2.2.0 |
| net.snowflake | snowflake-ingest-sdk | 0.9.6 |
| net.sourceforge.f2j | arpack_combined_all | 0.1 |
| org.acplt.remotetea | remotetea-oncrpc | 1.1.2 |
| org.antlr | ST4 | 4.0.4 |
| org.antlr | antlr-runtime | 3.5.2 |
| org.antlr | antlr4-runtime | 4.13.1 |
| org.antlr | stringtemplate | 3.2.1 |
| org.apache.ant | ant | 1.10.11 |
| org.apache.ant | ant-jsch | 1.10.11 |
| org.apache.ant | ant-launcher | 1.10.11 |
| org.apache.arrow | arrow-compression | 18.3.0 |
| org.apache.arrow | arrow-format | 18.3.0 |
| org.apache.arrow | arrow-memory-core | 18.3.0 |
| org.apache.arrow | arrow-memory-netty | 18.3.0 |
| org.apache.arrow | arrow-memory-netty-buffer-patch | 18.3.0 |
| org.apache.arrow | arrow-vector | 18.3.0 |
| org.apache.avro | avro | 1.12.1 |
| org.apache.avro | avro-ipc | 1.12.1 |
| org.apache.avro | avro-mapred | 1.12.1 |
| org.apache.commons | commons-collections4 | 4.5.0 |
| org.apache.commons | commons-compress | 1.28.0 |
| org.apache.commons | commons-configuration2 | 2.11.0 |
| org.apache.commons | commons-crypto | 1.1.0 |
| org.apache.commons | commons-lang3 | 3.19.0 |
| org.apache.commons | commons-math3 | 3.6.1 |
| org.apache.commons | commons-text | 1.14.0 |
| org.apache.curator | curator-client | 5.9.0 |
| org.apache.curator | curator-framework | 5.9.0 |
| org.apache.curator | curator-recipes | 5.9.0 |
| org.apache.datasketches | datasketches-java | 6.2.0 |
| org.apache.datasketches | datasketches-memory | 3.0.2 |
| org.apache.derby | derby | 10.14.2.0 |
| org.apache.hadoop | hadoop-client-runtime | 3.4.2 |
| org.apache.hive | hive-beeline | 2.3.10 |
| org.apache.hive | hive-cli | 2.3.10 |
| org.apache.hive | hive-jdbc | 2.3.10 |
| org.apache.hive | hive-llap-client | 2.3.10 |
| org.apache.hive | hive-llap-common | 2.3.10 |
| org.apache.hive | hive-serde | 2.3.10 |
| org.apache.hive | hive-shims | 2.3.10 |
| org.apache.hive | hive-storage-api | 2.8.1 |
| org.apache.hive.shims | hive-shims-0.23 | 2.3.10 |
| org.apache.hive.shims | hive-shims-common | 2.3.10 |
| org.apache.hive.shims | hive-shims-scheduler | 2.3.10 |
| org.apache.httpcomponents | httpclient | 4.5.14 |
| org.apache.httpcomponents | httpcore | 4.4.16 |
| org.apache.ivy | ivy | 2.5.3 |
| org.apache.logging.log4j | log4j-1.2-api | 2.24.3 |
| org.apache.logging.log4j | log4j-api | 2.24.3 |
| org.apache.logging.log4j | log4j-core | 2.24.3 |
| org.apache.logging.log4j | log4j-layout-template-json | 2.24.3 |
| org.apache.logging.log4j | log4j-slf4j2-impl | 2.24.3 |
| org.apache.orc | orc-core | 2.2.0-shaded-protobuf |
| org.apache.orc | orc-format | 1.1.1-shaded-protobuf |
| org.apache.orc | orc-mapreduce | 2.2.0-shaded-protobuf |
| org.apache.orc | orc-shims | 2.2.0 |
| org.apache.poi | poi | 5.4.1 |
| org.apache.poi | poi-ooxml | 5.4.1 |
| org.apache.poi | poi-ooxml-full | 5.4.1 |
| org.apache.poi | poi-ooxml-lite | 5.4.1 |
| org.apache.thrift | libfb303 | 0.9.3 |
| org.apache.thrift | libthrift | 0.16.0 |
| org.apache.ws.xmlschema | xmlschema-core | 2.3.1 |
| org.apache.xbean | xbean-asm9-shaded | 4.28 |
| org.apache.xmlbeans | xmlbeans | 5.3.0 |
| org.apache.yetus | audience-annotations | 0.13.0 |
| org.apache.zookeeper | zookeeper | 3.9.4 |
| org.apache.zookeeper | zookeeper-jute | 3.9.4 |
| org.checkerframework | checker-qual | 3.43.0 |
| org.codehaus.janino | commons-compiler | 3.0.16 |
| org.codehaus.janino | janino | 3.0.16 |
| org.datanucleus | datanucleus-api-jdo | 4.2.4 |
| org.datanucleus | datanucleus-core | 4.1.17 |
| org.datanucleus | datanucleus-rdbms | 4.1.19 |
| org.datanucleus | javax.jdo | 3.2.0-m3 |
| org.eclipse.jetty | jetty-alpn-client | 10.0.26 |
| org.eclipse.jetty | jetty-client | 10.0.26 |
| org.eclipse.jetty | jetty-http | 10.0.26 |
| org.eclipse.jetty | jetty-io | 10.0.26 |
| org.eclipse.jetty | jetty-jndi | 10.0.26 |
| org.eclipse.jetty | jetty-plus | 10.0.26 |
| org.eclipse.jetty | jetty-proxy | 10.0.26 |
| org.eclipse.jetty | jetty-security | 10.0.26 |
| org.eclipse.jetty | jetty-server | 10.0.26 |
| org.eclipse.jetty | jetty-servlet | 10.0.26 |
| org.eclipse.jetty | jetty-servlets | 10.0.26 |
| org.eclipse.jetty | jetty-util | 10.0.26 |
| org.eclipse.jetty | jetty-webapp | 10.0.26 |
| org.eclipse.jetty | jetty-xml | 10.0.26 |
| org.fusesource.leveldbjni | leveldbjni-all | 1.8 |
| org.glassfish.hk2 | hk2-api | 2.6.1 |
| org.glassfish.hk2 | hk2-locator | 2.6.1 |
| org.glassfish.hk2 | hk2-utils | 2.6.1 |
| org.glassfish.hk2 | osgi-resource-locator | 1.0.3 |
| org.glassfish.hk2.external | aopalliance-repackaged | 2.6.1 |
| org.glassfish.hk2.external | jakarta.inject | 2.6.1 |
| org.glassfish.jersey.containers | jersey-container-servlet | 2.41 |
| org.glassfish.jersey.containers | jersey-container-servlet-core | 2.41 |
| org.glassfish.jersey.core | jersey-client | 2.41 |
| org.glassfish.jersey.core | jersey-common | 2.41 |
| org.glassfish.jersey.core | jersey-server | 2.41 |
| org.glassfish.jersey.inject | jersey-hk2 | 2.41 |
| org.hibernate.validator | hibernate-validator | 6.2.5.Final |
| org.ini4j | ini4j | 0.5.4 |
| org.javassist | javassist | 3.29.2-GA |
| org.jboss.logging | jboss-logging | 3.4.1.Final |
| org.jdbi | jdbi | 2.63.1 |
| org.jline | jline | 3.27.1-jdk8 |
| org.joda | joda-convert | 1.7 |
| org.json4s | json4s-ast_2.13 | 4.0.7 |
| org.json4s | json4s-core_2.13 | 4.0.7 |
| org.json4s | json4s-jackson-core_2.13 | 4.0.7 |
| org.json4s | json4s-jackson_2.13 | 4.0.7 |
| org.json4s | json4s-scalap_2.13 | 4.0.7 |
| org.locationtech.jts | jts-core | 1.20.0 |
| org.lz4 | lz4-java | 1.8.0-databricks-1 |
| org.mlflow | mlflow-spark_2.13 | 2.22.1 |
| org.objenesis | objenesis | 3.4 |
| org.postgresql | postgresql | 42.6.1 |
| org.roaringbitmap | RoaringBitmap | 1.2.1 |
| org.rocksdb | rocksdbjni | 9.8.4 |
| org.rosuda.REngine | REngine | 2.1.0 |
| org.scala-lang | scala-compiler_2.13 | 2.13.16 |
| org.scala-lang | scala-library_2.13 | 2.13.16 |
| org.scala-lang | scala-reflect_2.13 | 2.13.16 |
| org.scala-lang.modules | scala-collection-compat_2.13 | 2.11.0 |
| org.scala-lang.modules | scala-java8-compat_2.13 | 0.9.1 |
| org.scala-lang.modules | scala-parallel-collections_2.13 | 1.2.0 |
| org.scala-lang.modules | scala-parser-combinators_2.13 | 2.4.0 |
| org.scala-lang.modules | scala-xml_2.13 | 2.4.0 |
| org.scala-sbt | test-interface | 1.0 |
| org.scalacheck | scalacheck_2.13 | 1.18.0 |
| org.scalactic | scalactic_2.13 | 3.2.19 |
| org.scalanlp | breeze-macros_2.13 | 2.1.0 |
| org.scalanlp | breeze_2.13 | 2.1.0 |
| org.scalatest | scalatest-compatible | 3.2.19 |
| org.scalatest | scalatest-core_2.13 | 3.2.19 |
| org.scalatest | scalatest-diagrams_2.13 | 3.2.19 |
| org.scalatest | scalatest-featurespec_2.13 | 3.2.19 |
| org.scalatest | scalatest-flatspec_2.13 | 3.2.19 |
| org.scalatest | scalatest-freespec_2.13 | 3.2.19 |
| org.scalatest | scalatest-funspec_2.13 | 3.2.19 |
| org.scalatest | scalatest-funsuite_2.13 | 3.2.19 |
| org.scalatest | scalatest-matchers-core_2.13 | 3.2.19 |
| org.scalatest | scalatest-mustmatchers_2.13 | 3.2.19 |
| org.scalatest | scalatest-propspec_2.13 | 3.2.19 |
| org.scalatest | scalatest-refspec_2.13 | 3.2.19 |
| org.scalatest | scalatest-shouldmatchers_2.13 | 3.2.19 |
| org.scalatest | scalatest-wordspec_2.13 | 3.2.19 |
| org.scalatest | scalatest_2.13 | 3.2.19 |
| org.slf4j | jcl-over-slf4j | 2.0.16 |
| org.slf4j | jul-to-slf4j | 2.0.16 |
| org.slf4j | slf4j-api | 2.0.16 |
| org.slf4j | slf4j-simple | 1.7.25 |
| org.threeten | threeten-extra | 1.8.0 |
| org.tukaani | xz | 1.10 |
| org.typelevel | algebra_2.13 | 2.8.0 |
| org.typelevel | cats-kernel_2.13 | 2.8.0 |
| org.typelevel | spire-macros_2.13 | 0.18.0 |
| org.typelevel | spire-platform_2.13 | 0.18.0 |
| org.typelevel | spire-util_2.13 | 0.18.0 |
| org.typelevel | spire_2.13 | 0.18.0 |
| org.wildfly.openssl | wildfly-openssl | 1.1.3.Final |
| org.xerial | sqlite-jdbc | 3.42.0.0 |
| org.xerial.snappy | snappy-java | 1.1.10.3 |
| org.yaml | snakeyaml | 2.0 |
| oro | oro | 2.0.8 |
| pl.edu.icm | JLargeArrays | 1.5 |
| software.amazon.cryptools | AmazonCorrettoCryptoProvider | 2.5.0-linux-x86_64 |
| stax | stax-api | 1.0.1 |