Jaa


Databricks Runtime 10.2 (EoS)

Note

Support for this Databricks Runtime version has ended. For the end-of-support date, see End-of-support history. For all supported Databricks Runtime versions, see Databricks Runtime release notes versions and compatibility.

The following release notes provide information about Databricks Runtime 10.2 and Databricks Runtime 10.2 Photon, powered by Apache Spark 3.2.0. Databricks released this version in December 2021. Photon is in Public Preview.

New features and improvements

Use Files in Repos with Spark Streaming

You can now access Files in Repos from Spark Streaming.

Databricks Utilities adds an update mount command

The new dbutils.fs.updateMount command is similar to dbutils.fs.mount, but this new command updates an existing mount point instead of creating a new one. See updateMount command (dbutils.fs.updateMount).

Delta Lake supports renaming of columns (Public Preview)

Delta Lake now supports column renaming. To rename the column of a Delta table, you must upgrade your Delta table’s version and then set the column mapping mode to name mapping. See Rename and drop columns with Delta Lake column mapping.

Delta Lake table history adds Databricks Runtime version information

When a write is now made to a Delta table, information about the Databricks Runtime version that made the write is also recorded. To get the version information, run the DESCRIBE HISTORY command on the table and then see the engineInfo column in the output. See DESCRIBE HISTORY and Work with Delta Lake table history.

Databricks Utilities adds latest modification time information when directories are listed

The dbutils.fs.ls command for listing the contents of a directory now outputs a modificationTime field. This field contains the latest modification time for each listed file or directory. See ls command (dbutils.fs.ls).

Spark SQL adds an approx_top_k function to return top items and their counts

The new approx_top_k function returns the most frequently occurring item values, along with their approximate counts, for the specified expression. Some examples include:

df = spark.sql('SELECT approx_top_k(col, 10, 100) FROM VALUES (0), (1), (1), (2), (2), (2) AS tab(col)')

display(df)
import pyspark.sql.functions as F

df = spark.createDataFrame([
       (0,), (1,), (1,), (2,), (2,), (2,)
     ]).select(F.expr('approx_top_k(_1, 10)'))

display(df)

The result of running either example is:

[{"item": 2, "count": 3}, {"item": 1, "count": 2}, {"item": 0, "count": 1}]

See approx_top_k aggregate function.

Trigger.AvailableNow now available in Python as well as Scala

This change supports the Delta source streaming queries Trigger.AvailableNow in Python as well as Scala. Databricks Runtime 10.1 introduced support for this trigger in Scala. Databricks Runtime 10.2 now supports this trigger in Python as well. See coverage of Trigger.AvailableNow in What is Auto Loader?.

Auto Loader now supports schema evolution for Avro (Public Preview)

See Configure schema inference and evolution in Auto Loader.

Library upgrades

  • Upgraded Python libraries:
    • filelock from 3.3.1 to 3.4.0
    • plotly from 5.3.0 to 5.3.1
  • Upgraded R libraries:
    • base from 4.1.1 to 4.1.2
    • compiler from 4.1.1 to 4.1.2
    • datasets from 4.1.1 to 4.1.2
    • graphics from 4.1.1 to 4.1.2
    • grDevices from 4.1.1 to 4.1.2
    • grid from 4.1.1 to 4.1.2
    • methods from 4.1.1 to 4.1.2
    • parallel from 4.1.1 to 4.1.2
    • Rserve from 1.8-8 to 1.8-10
    • splines from 4.1.1 to 4.1.2
    • stats from 4.1.1 to 4.1.2
    • stats4 from 4.1.1 to 4.1.2
    • tcltk from 4.1.1 to 4.1.2
    • tools from 4.1.1 to 4.1.2
    • utils from 4.1.1 to 4.1.2
    • backports from 1.2.1 to 1.4.0
    • broom from 0.7.9 to 0.7.10
    • cli from 3.0.1 to 3.1.0
    • cpp11 from 0.4.0 to 0.4.1
    • crayon from 1.4.1 to 1.4.2
    • credentials from 1.3.1 to 1.3.2
    • future from 1.22.1 to 1.23.0
    • gert from 1.4.1 to 1.4.3
    • glmnet from 4.1-2 to 4.1-3
    • glue from 1.4.2 to 1.5.0
    • memoise from 2.0.0 to 2.0.1
    • parallelly from 1.28.1 to 1.29.0
    • readr from 2.0.2 to 2.1.0
    • sessioninfo from 1.1.1 to 1.2.1
    • stringi from 1.7.5 to 1.7.6
    • tibble from 3.1.5 to 3.1.6
    • tinytex from 0.34 to 0.35
    • tzdb from 0.1.2 to 0.2.0
    • usethis from 2.1.2 to 2.1.3
    • uuid from 0.1-4 to 1.0-3
    • vroom from 1.5.5 to 1.5.6
    • xfun from 0.27 to 0.28
  • Upgraded Java libraries:
    • com.fasterxml.jackson from 2.12.2 to 2.12.3
    • io.airlift.aircompressor from 0.19 to 0.21
    • io.netty.netty-all from 4.1.63.Final to 4.1.68.Final
    • joda-time.joda-time from 2.10.5 to 2.10.10
    • org.apache.commons.commons-compress from 1.20 to 1.21
    • org.apache.orc from 1.6.10 to 1.6.12
    • org.eclipse.jetty from 9.4.40.v20210413 to 9.4.43.v20210629
    • org.scalanlp.breeze-macros_2.12 from 1.0 to 1.2
    • org.scalanlp.breeze_2.12 from 1.0 to 1.2
    • org.org.typelevel.algebra_2.12 from 2.0.0 to 2.0.1
    • org.typelevel.cats-kernel_2.12 from 2.0.0 to 2.0.1

Apache Spark

Databricks Runtime 10.2 includes Apache Spark 3.2.0. This release includes all Spark fixes and improvements included in Databricks Runtime 10.1 (EoS), as well as the following additional bug fixes and improvements made to Spark:

  • [SPARK-37447] [SQL] Cache LogicalPlan.isStreaming() result in a lazy val
  • [SPARK-37468] [SQL] Support ANSI intervals and TimestampNTZ for UnionEstimation
  • [SPARK-37464] [SQL] SCHEMA and DATABASE should simply be aliases of NAMESPACE
  • [SPARK-37452] [SQL] Char and Varchar break backward compatibility between v3.1 and v2
  • [SPARK-36861] [SQL] Use yyyy-MM-dd as the date pattern in partition discovery
  • [SPARK-37456] [SQL] CREATE NAMESPACE should qualify location for v2 command
  • [SPARK-37357] [SQL] Add small partition factor for rebalance partitions
  • [SPARK-37446] [SQL] Use reflection for getWithoutRegisterFns to allow different Hive versions for building
  • [SPARK-37444] [SQL] ALTER NAMESPACE … SET LOCATION should handle empty location consistently across v1 and v2 command
  • [SPARK-36357] [SQL] Support pushdown Timestamp without time zone for orc
  • [SPARK-36346] [SQL] Support TimestampNTZ type in Orc file source
  • [SPARK-36646] [SQL] Push down group by partition column for aggregate
  • [SPARK-34960] [SQL] Aggregate push down for ORC
  • [SPARK-37386] [SQL] Simplify OptimizeSkewedJoin to not run the cost evaluator
  • [SPARK-36073] [SQL] EquivalentExpressions fixes and improvements
  • [SPARK-35352] [SQL] Add code-gen for full outer sort merge join
  • [SPARK-37292] [SQL] Removes outer join if it only has DISTINCT on streamed side with alias
  • [SPARK-37371] [SQL] UnionExec should support columnar if all children support columnar
  • [SPARK-36645] [SQL] Aggregate (Min/Max/Count) push down for Parquet
  • [SPARK-37379] [SQL] Add tree pattern pruning to CTESubstitution rule
  • [SPARK-37332] [SQL] Allow ANSI intervals in ALTER TABLE .. ADD COLUMNS
  • [SPARK-37224] [SS] Optimize write path on RocksDB state store provider
  • [SPARK-37304] [SQL] Allow ANSI intervals in v2 ALTER TABLE .. REPLACE COLUMNS
  • [SPARK-36924] [SQL] CAST between ANSI intervals and IntegralType
  • [SPARK-37289] [SQL] Remove partitionSchemaOption function
  • [SPARK-37270] [SQL] Fix push foldable into CaseWhen branches if elseValue is empty
  • [SPARK-35345] [SQL] Add Parquet tests to BloomFilterBenchmark
  • [SPARK-37341] [SQL] Avoid unnecessary buffer and copy in full outer sort merge join
  • [SPARK-37266] [SQL] View text can only be SELECT queries
  • [SPARK-37288] [PYTHON][3.2] Backport since annotation update
  • [SPARK-37262] [SQL] Don’t log empty aggregate and group by in JDBCScan
  • [SPARK-37283] [SQL] Don’t try to store a V1 table which contains ANSI intervals in Hive compatible format
  • [SPARK-37221] [SQL] The collect-like API in SparkPlan should support columnar output
  • [SPARK-37202] [SQL] Use AnalysisContext to track referred temp functions
  • [SPARK-37023] [CORE] Avoid fetching merge status when shuffleMergeEnabled is false for a shuffleDependency during retry
  • [SPARK-37307] [CORE] Don’t obtain JDBC connection for empty partitions
  • [SPARK-37267] [SQL] OptimizeSkewInRebalancePartitions support optimize non-root node
  • [SPARK-36799] [SQL] Pass queryExecution name in CLI
  • [SPARK-37261] [SQL] Allow adding partitions with ANSI intervals in DSv2
  • [SPARK-37203] [SQL] Fix NotSerializableException when observe with TypedImperativeAggregate
  • [SPARK-37199] [SQL] Add deterministic field to QueryPlan
  • [SPARK-37253] [PYTHON] try_simplify_traceback should not fail when tb_frame.f_lineno is None
  • [SPARK-37214] [SQL] Fail query analysis earlier with invalid identifiers
  • [SPARK-37240] [SQL] Handle ANSI intervals by ColumnVectorUtils.populate()
  • [SPARK-35437] [SQL] Use expressions to filter Hive partitions at client side
  • [SPARK-32940] [SQL] Collect, first and last should be deterministic aggregate functions
  • [SPARK-37231] [SQL] Dynamic writes/reads of ANSI interval partitions
  • [SPARK-37038] [SQL] DSV2 Sample Push Down
  • [SPARK-36665] [SQL] Add more Not operator simplifications
  • [SPARK-37150] [SQL] Migrate DESCRIBE NAMESPACE to use V2 command by default
  • [SPARK-37196] [SQL] HiveDecimal enforcePrecisionScale failed return null
  • [SPARK-37212] [SQL] Improve the implement of aggregate pushdown.
  • [SPARK-37190] [SQL] Improve error messages for Cast under ANSI mode
  • [SPARK-37161] [SQL] RowToColumnConverter support AnsiIntervalType
  • [SPARK-37160] [SQL] Add a config to optionally disable padding for char type
  • [SPARK-36554] [SQL][PYTHON] Expose make_date expression in functions.scala
  • [SPARK-37200] [SQL] Support drop index for Data Source V2
  • [SPARK-37179] [SQL] ANSI mode: Add a config to allow casting between Datetime and Numeric
  • [SPARK-37168] [SQL] Improve error messages for SQL functions and operators under ANSI mode
  • [SPARK-36935] [SQL] Extend ParquetSchemaConverter to compute Parquet repetition & definition level
  • [SPARK-37066] [SQL] Improve error message to show file path when failed to read next file
  • [SPARK-32567] [SQL] Add code-gen for full outer shuffled hash join
  • [SPARK-37176] [SQL] Sync JsonInferSchema#infer method’s exception handle logic with JacksonParser#parse method
  • [SPARK-37149] [SQL] Improve error messages for arithmetic overflow exceptions under ANSI mode
  • [SPARK-37136] [SQL] Remove code about hive buildin function but not implement in spark
  • [SPARK-16280] [SPARK-37082] [SQL] Implements histogram_numeric aggregation function which supports partial aggregation.
  • [SPARK-36975] [SQL] Correct the hive client calls‘s metrics in HiveClientImpl

Maintenance updates

See Databricks Runtime 10.2 maintenance updates.

System environment

  • Operating System: Ubuntu 20.04.3 LTS
  • Java: Zulu 8.58.0.13-CA-linux64
  • Scala: 2.12.14
  • Python: 3.8.10
  • R: 4.1.2
  • Delta Lake: 1.1.0

Installed Python libraries

Library Version Library Version Library Version
Antergos Linux 2015.10 (ISO-Rolling) appdirs 1.4.4 argon2-cffi 20.1.0
async-generator 1.10 attrs 20.3.0 backcall 0.2.0
bidict 0.21.4 bleach 3.3.0 boto3 1.16.7
botocore 1.19.7 certifi 2020.12.5 cffi 1.14.5
chardet 4.0.0 cycler 0.10.0 Cython 0.29.23
dbus-python 1.2.16 decorator 5.0.6 defusedxml 0.7.1
distlib 0.3.3 distro-info 0.23ubuntu1 entrypoints 0.3
facets-overview 1.0.0 filelock 3.4.0 idna 2.10
ipykernel 5.3.4 ipython 7.22.0 ipython-genutils 0.2.0
ipywidgets 7.6.3 jedi 0.17.2 Jinja2 2.11.3
jmespath 0.10.0 joblib 1.0.1 jsonschema 3.2.0
jupyter-client 6.1.12 jupyter-core 4.7.1 jupyterlab-pygments 0.1.2
jupyterlab-widgets 1.0.0 kiwisolver 1.3.1 koalas 1.8.2
MarkupSafe 2.0.1 matplotlib 3.4.2 mistune 0.8.4
nbclient 0.5.3 nbconvert 6.0.7 nbformat 5.1.3
nest-asyncio 1.5.1 notebook 6.3.0 numpy 1.19.2
packaging 20.9 pandas 1.2.4 pandocfilters 1.4.3
parso 0.7.0 patsy 0.5.1 pexpect 4.8.0
pickleshare 0.7.5 Pillow 8.2.0 pip 21.0.1
plotly 5.3.1 prometheus-client 0.10.1 prompt-toolkit 3.0.17
protobuf 3.17.2 psycopg2 2.8.5 ptyprocess 0.7.0
pyarrow 4.0.0 pycparser 2.20 Pygments 2.8.1
PyGObject 3.36.0 pyparsing 2.4.7 pyrsistent 0.17.3
python-apt 2.0.0+ubuntu0.20.4.6 python-dateutil 2.8.1 python-engineio 4.3.0
python-socketio 5.4.1 pytz 2020.5 pyzmq 20.0.0
requests 2.25.1 requests-unixsocket 0.2.0 s3transfer 0.3.7
scikit-learn 0.24.1 scipy 1.6.2 seaborn 0.11.1
Send2Trash 1.5.0 setuptools 52.0.0 six 1.15.0
ssh-import-id 5.10 statsmodels 0.12.2 tenacity 8.0.1
terminado 0.9.4 testpath 0.4.4 threadpoolctl 2.1.0
tornado 6.1 traitlets 5.0.5 unattended-upgrades 0.1
urllib3 1.25.11 virtualenv 20.4.1 wcwidth 0.2.5
webencodings 0.5.1 wheel 0.36.2 widgetsnbextension 3.5.1

Installed R libraries

R libraries are installed from the Microsoft CRAN snapshot on 2021-11-30.

Library Version Library Version Library Version
askpass 1.1 assertthat 0.2.1 backports 1.4.0
base 4.1.2 base64enc 0.1-3 bit 4.0.4
bit64 4.0.5 blob 1.2.2 boot 1.3-28
brew 1.0-6 brio 1.1.2 broom 0.7.10
bslib 0.3.1 cachem 1.0.6 callr 3.7.0
caret 6.0-90 cellranger 1.1.0 chron 2.3-56
class 7.3-19 cli 3.1.0 clipr 0.7.1
cluster 2.1.2 codetools 0.2-18 colorspace 2.0-2
commonmark 1.7 compiler 4.1.2 config 0.3.1
cpp11 0.4.1 crayon 1.4.2 credentials 1.3.2
curl 4.3.2 data.table 1.14.2 datasets 4.1.2
DBI 1.1.1 dbplyr 2.1.1 desc 1.4.0
devtools 2.4.2 diffobj 0.3.5 digest 0.6.28
dplyr 1.0.7 dtplyr 1.1.0 e1071 1.7-9
ellipsis 0.3.2 evaluate 0.14 fansi 0.5.0
farver 2.1.0 fastmap 1.1.0 fontawesome 0.2.2
forcats 0.5.1 foreach 1.5.1 foreign 0.8-81
forge 0.2.0 fs 1.5.0 future 1.23.0
future.apply 1.8.1 gargle 1.2.0 generics 0.1.1
gert 1.4.3 ggplot2 3.3.5 gh 1.3.0
gitcreds 0.1.1 glmnet 4.1-3 globals 0.14.0
glue 1.5.0 googledrive 2.0.0 googlesheets4 1.0.0
gower 0.2.2 graphics 4.1.2 grDevices 4.1.2
grid 4.1.2 gridExtra 2.3 gsubfn 0.7
gtable 0.3.0 haven 2.4.3 highr 0.9
hms 1.1.1 htmltools 0.5.2 htmlwidgets 1.5.4
httpuv 1.6.3 httr 1.4.2 hwriter 1.3.2
hwriterPlus 1.0-3 ids 1.0.1 ini 0.3.1
ipred 0.9-12 isoband 0.2.5 iterators 1.0.13
jquerylib 0.1.4 jsonlite 1.7.2 KernSmooth 2.23-20
knitr 1.36 labeling 0.4.2 later 1.3.0
lattice 0.20-45 lava 1.6.10 lifecycle 1.0.1
listenv 0.8.0 lubridate 1.8.0 magrittr 2.0.1
markdown 1.1 MASS 7.3-54 Matrix 1.3-4
memoise 2.0.1 methods 4.1.2 mgcv 1.8-38
mime 0.12 ModelMetrics 1.2.2.2 modelr 0.1.8
munsell 0.5.0 nlme 3.1-152 nnet 7.3-16
numDeriv 2016.8-1.1 openssl 1.4.5 parallel 4.1.2
parallelly 1.29.0 pillar 1.6.4 pkgbuild 1.2.0
pkgconfig 2.0.3 pkgload 1.2.3 plogr 0.2.0
plyr 1.8.6 praise 1.0.0 prettyunits 1.1.1
pROC 1.18.0 processx 3.5.2 prodlim 2019.11.13
progress 1.2.2 progressr 0.9.0 promises 1.2.0.1
proto 1.0.0 proxy 0.4-26 ps 1.6.0
purrr 0.3.4 r2d3 0.2.5 R6 2.5.1
randomForest 4.6-14 rappdirs 0.3.3 rcmdcheck 1.4.0
RColorBrewer 1.1-2 Rcpp 1.0.7 RcppEigen 0.3.3.9.1
readr 2.1.0 readxl 1.3.1 recipes 0.1.17
rematch 1.0.1 rematch2 2.1.2 remotes 2.4.1
reprex 2.0.1 reshape2 1.4.4 rlang 0.4.12
rmarkdown 2.11 RODBC 1.3-19 roxygen2 7.1.2
rpart 4.1-15 rprojroot 2.0.2 Rserve 1.8-10
RSQLite 2.2.8 rstudioapi 0.13 rversions 2.1.1
rvest 1.0.2 sass 0.4.0 scales 1.1.1
selectr 0.4-2 sessioninfo 1.2.1 shape 1.4.6
shiny 1.7.1 sourcetools 0.1.7 sparklyr 1.7.2
SparkR 3.2.0 spatial 7.3-11 splines 4.1.2
sqldf 0.4-11 SQUAREM 2021.1 stats 4.1.2
stats4 4.1.2 stringi 1.7.6 stringr 1.4.0
survival 3.2-13 sys 3.4 tcltk 4.1.2
TeachingDemos 2.10 testthat 3.1.0 tibble 3.1.6
tidyr 1.1.4 tidyselect 1.1.1 tidyverse 1.3.1
timeDate 3043.102 tinytex 0.35 tools 4.1.2
tzdb 0.2.0 usethis 2.1.3 utf8 1.2.2
utils 4.1.2 uuid 1.0-3 vctrs 0.3.8
viridisLite 0.4.0 vroom 1.5.6 waldo 0.3.1
whisker 0.4 withr 2.4.2 xfun 0.28
xml2 1.3.2 xopen 1.0.0 xtable 1.8-4
yaml 2.2.1 zip 2.2.0

Installed Java and Scala libraries (Scala 2.12 cluster version)

Group ID Artifact ID Version
antlr antlr 2.7.7
com.amazonaws amazon-kinesis-client 1.12.0
com.amazonaws aws-java-sdk-autoscaling 1.11.655
com.amazonaws aws-java-sdk-cloudformation 1.11.655
com.amazonaws aws-java-sdk-cloudfront 1.11.655
com.amazonaws aws-java-sdk-cloudhsm 1.11.655
com.amazonaws aws-java-sdk-cloudsearch 1.11.655
com.amazonaws aws-java-sdk-cloudtrail 1.11.655
com.amazonaws aws-java-sdk-cloudwatch 1.11.655
com.amazonaws aws-java-sdk-cloudwatchmetrics 1.11.655
com.amazonaws aws-java-sdk-codedeploy 1.11.655
com.amazonaws aws-java-sdk-cognitoidentity 1.11.655
com.amazonaws aws-java-sdk-cognitosync 1.11.655
com.amazonaws aws-java-sdk-config 1.11.655
com.amazonaws aws-java-sdk-core 1.11.655
com.amazonaws aws-java-sdk-datapipeline 1.11.655
com.amazonaws aws-java-sdk-directconnect 1.11.655
com.amazonaws aws-java-sdk-directory 1.11.655
com.amazonaws aws-java-sdk-dynamodb 1.11.655
com.amazonaws aws-java-sdk-ec2 1.11.655
com.amazonaws aws-java-sdk-ecs 1.11.655
com.amazonaws aws-java-sdk-efs 1.11.655
com.amazonaws aws-java-sdk-elasticache 1.11.655
com.amazonaws aws-java-sdk-elasticbeanstalk 1.11.655
com.amazonaws aws-java-sdk-elasticloadbalancing 1.11.655
com.amazonaws aws-java-sdk-elastictranscoder 1.11.655
com.amazonaws aws-java-sdk-emr 1.11.655
com.amazonaws aws-java-sdk-glacier 1.11.655
com.amazonaws aws-java-sdk-glue 1.11.655
com.amazonaws aws-java-sdk-iam 1.11.655
com.amazonaws aws-java-sdk-importexport 1.11.655
com.amazonaws aws-java-sdk-kinesis 1.11.655
com.amazonaws aws-java-sdk-kms 1.11.655
com.amazonaws aws-java-sdk-lambda 1.11.655
com.amazonaws aws-java-sdk-logs 1.11.655
com.amazonaws aws-java-sdk-machinelearning 1.11.655
com.amazonaws aws-java-sdk-opsworks 1.11.655
com.amazonaws aws-java-sdk-rds 1.11.655
com.amazonaws aws-java-sdk-redshift 1.11.655
com.amazonaws aws-java-sdk-route53 1.11.655
com.amazonaws aws-java-sdk-s3 1.11.655
com.amazonaws aws-java-sdk-ses 1.11.655
com.amazonaws aws-java-sdk-simpledb 1.11.655
com.amazonaws aws-java-sdk-simpleworkflow 1.11.655
com.amazonaws aws-java-sdk-sns 1.11.655
com.amazonaws aws-java-sdk-sqs 1.11.655
com.amazonaws aws-java-sdk-ssm 1.11.655
com.amazonaws aws-java-sdk-storagegateway 1.11.655
com.amazonaws aws-java-sdk-sts 1.11.655
com.amazonaws aws-java-sdk-support 1.11.655
com.amazonaws aws-java-sdk-swf-libraries 1.11.22
com.amazonaws aws-java-sdk-workspaces 1.11.655
com.amazonaws jmespath-java 1.11.655
com.chuusai shapeless_2.12 2.3.3
com.clearspring.analytics stream 2.9.6
com.databricks Rserve 1.8-3
com.databricks jets3t 0.7.1-0
com.databricks.scalapb compilerplugin_2.12 0.4.15-10
com.databricks.scalapb scalapb-runtime_2.12 0.4.15-10
com.esotericsoftware kryo-shaded 4.0.2
com.esotericsoftware minlog 1.3.0
com.fasterxml classmate 1.3.4
com.fasterxml.jackson.core jackson-annotations 2.12.3
com.fasterxml.jackson.core jackson-core 2.12.3
com.fasterxml.jackson.core jackson-databind 2.12.3
com.fasterxml.jackson.dataformat jackson-dataformat-cbor 2.12.3
com.fasterxml.jackson.datatype jackson-datatype-joda 2.12.3
com.fasterxml.jackson.module jackson-module-paranamer 2.12.3
com.fasterxml.jackson.module jackson-module-scala_2.12 2.12.3
com.github.ben-manes.caffeine caffeine 2.3.4
com.github.fommil jniloader 1.1
com.github.fommil.netlib core 1.1.2
com.github.fommil.netlib native_ref-java 1.1
com.github.fommil.netlib native_ref-java-natives 1.1
com.github.fommil.netlib native_system-java 1.1
com.github.fommil.netlib native_system-java-natives 1.1
com.github.fommil.netlib netlib-native_ref-linux-x86_64-natives 1.1
com.github.fommil.netlib netlib-native_system-linux-x86_64-natives 1.1
com.github.luben zstd-jni 1.5.0-4
com.github.wendykierp JTransforms 3.1
com.google.code.findbugs jsr305 3.0.0
com.google.code.gson gson 2.8.6
com.google.crypto.tink tink 1.6.0
com.google.flatbuffers flatbuffers-java 1.9.0
com.google.guava guava 15.0
com.google.protobuf protobuf-java 2.6.1
com.h2database h2 1.4.195
com.helger profiler 1.1.1
com.jcraft jsch 0.1.50
com.jolbox bonecp 0.8.0.RELEASE
com.lihaoyi sourcecode_2.12 0.1.9
com.microsoft.azure azure-data-lake-store-sdk 2.3.9
com.microsoft.sqlserver mssql-jdbc 9.2.1.jre8
com.ning compress-lzf 1.0.3
com.sun.istack istack-commons-runtime 3.0.8
com.sun.mail javax.mail 1.5.2
com.tdunning json 1.8
com.thoughtworks.paranamer paranamer 2.8
com.trueaccord.lenses lenses_2.12 0.4.12
com.twitter chill-java 0.10.0
com.twitter chill_2.12 0.10.0
com.twitter util-app_2.12 7.1.0
com.twitter util-core_2.12 7.1.0
com.twitter util-function_2.12 7.1.0
com.twitter util-jvm_2.12 7.1.0
com.twitter util-lint_2.12 7.1.0
com.twitter util-registry_2.12 7.1.0
com.twitter util-stats_2.12 7.1.0
com.typesafe config 1.2.1
com.typesafe.scala-logging scala-logging_2.12 3.7.2
com.univocity univocity-parsers 2.9.1
com.zaxxer HikariCP 4.0.3
commons-cli commons-cli 1.2
commons-codec commons-codec 1.15
commons-collections commons-collections 3.2.2
commons-dbcp commons-dbcp 1.4
commons-fileupload commons-fileupload 1.3.3
commons-httpclient commons-httpclient 3.1
commons-io commons-io 2.8.0
commons-lang commons-lang 2.6
commons-logging commons-logging 1.1.3
commons-net commons-net 3.1
commons-pool commons-pool 1.5.4
dev.ludovic.netlib arpack 1.3.2
dev.ludovic.netlib blas 1.3.2
dev.ludovic.netlib lapack 1.3.2
hive-2.3__hadoop-3.2 jets3t-0.7 liball_deps_2.12
info.ganglia.gmetric4j gmetric4j 1.0.10
io.airlift aircompressor 0.21
io.delta delta-sharing-spark_2.12 0.2.0
io.dropwizard.metrics metrics-core 4.1.1
io.dropwizard.metrics metrics-graphite 4.1.1
io.dropwizard.metrics metrics-healthchecks 4.1.1
io.dropwizard.metrics metrics-jetty9 4.1.1
io.dropwizard.metrics metrics-jmx 4.1.1
io.dropwizard.metrics metrics-json 4.1.1
io.dropwizard.metrics metrics-jvm 4.1.1
io.dropwizard.metrics metrics-servlets 4.1.1
io.netty netty-all 4.1.68.Final
io.prometheus simpleclient 0.7.0
io.prometheus simpleclient_common 0.7.0
io.prometheus simpleclient_dropwizard 0.7.0
io.prometheus simpleclient_pushgateway 0.7.0
io.prometheus simpleclient_servlet 0.7.0
io.prometheus.jmx collector 0.12.0
jakarta.annotation jakarta.annotation-api 1.3.5
jakarta.servlet jakarta.servlet-api 4.0.3
jakarta.validation jakarta.validation-api 2.0.2
jakarta.ws.rs jakarta.ws.rs-api 2.1.6
javax.activation activation 1.1.1
javax.annotation javax.annotation-api 1.3.2
javax.el javax.el-api 2.2.4
javax.jdo jdo-api 3.0.1
javax.transaction jta 1.1
javax.transaction transaction-api 1.1
javax.xml.bind jaxb-api 2.2.2
javax.xml.stream stax-api 1.0-2
javolution javolution 5.5.1
jline jline 2.14.6
joda-time joda-time 2.10.10
log4j apache-log4j-extras 1.2.17
log4j log4j 1.2.17
maven-trees hive-2.3__hadoop-3.2 liball_deps_2.12
net.java.dev.jna jna 5.8.0
net.razorvine pyrolite 4.30
net.sf.jpam jpam 1.1
net.sf.opencsv opencsv 2.3
net.sf.supercsv super-csv 2.2.0
net.snowflake snowflake-ingest-sdk 0.9.6
net.snowflake snowflake-jdbc 3.13.3
net.snowflake spark-snowflake_2.12 2.9.0-spark_3.1
net.sourceforge.f2j arpack_combined_all 0.1
org.acplt.remotetea remotetea-oncrpc 1.1.2
org.antlr ST4 4.0.4
org.antlr antlr-runtime 3.5.2
org.antlr antlr4-runtime 4.8
org.antlr stringtemplate 3.2.1
org.apache.ant ant 1.9.2
org.apache.ant ant-jsch 1.9.2
org.apache.ant ant-launcher 1.9.2
org.apache.arrow arrow-format 2.0.0
org.apache.arrow arrow-memory-core 2.0.0
org.apache.arrow arrow-memory-netty 2.0.0
org.apache.arrow arrow-vector 2.0.0
org.apache.avro avro 1.10.2
org.apache.avro avro-ipc 1.10.2
org.apache.avro avro-mapred 1.10.2
org.apache.commons commons-compress 1.21
org.apache.commons commons-crypto 1.1.0
org.apache.commons commons-lang3 3.12.0
org.apache.commons commons-math3 3.4.1
org.apache.commons commons-text 1.6
org.apache.curator curator-client 2.13.0
org.apache.curator curator-framework 2.13.0
org.apache.curator curator-recipes 2.13.0
org.apache.derby derby 10.14.2.0
org.apache.hadoop hadoop-client-api 3.3.1-databricks
org.apache.hadoop hadoop-client-runtime 3.3.1
org.apache.hive hive-beeline 2.3.9
org.apache.hive hive-cli 2.3.9
org.apache.hive hive-jdbc 2.3.9
org.apache.hive hive-llap-client 2.3.9
org.apache.hive hive-llap-common 2.3.9
org.apache.hive hive-serde 2.3.9
org.apache.hive hive-shims 2.3.9
org.apache.hive hive-storage-api 2.7.2
org.apache.hive.shims hive-shims-0.23 2.3.9
org.apache.hive.shims hive-shims-common 2.3.9
org.apache.hive.shims hive-shims-scheduler 2.3.9
org.apache.htrace htrace-core4 4.1.0-incubating
org.apache.httpcomponents httpclient 4.5.13
org.apache.httpcomponents httpcore 4.4.12
org.apache.ivy ivy 2.5.0
org.apache.mesos mesos-shaded-protobuf 1.4.0
org.apache.orc orc-core 1.6.12
org.apache.orc orc-mapreduce 1.6.12
org.apache.orc orc-shims 1.6.12
org.apache.parquet parquet-column 1.12.0-databricks-0003
org.apache.parquet parquet-common 1.12.0-databricks-0003
org.apache.parquet parquet-encoding 1.12.0-databricks-0003
org.apache.parquet parquet-format-structures 1.12.0-databricks-0003
org.apache.parquet parquet-hadoop 1.12.0-databricks-0003
org.apache.parquet parquet-jackson 1.12.0-databricks-0003
org.apache.thrift libfb303 0.9.3
org.apache.thrift libthrift 0.12.0
org.apache.xbean xbean-asm9-shaded 4.20
org.apache.yetus audience-annotations 0.5.0
org.apache.zookeeper zookeeper 3.6.2
org.apache.zookeeper zookeeper-jute 3.6.2
org.checkerframework checker-qual 3.5.0
org.codehaus.jackson jackson-core-asl 1.9.13
org.codehaus.jackson jackson-mapper-asl 1.9.13
org.codehaus.janino commons-compiler 3.0.16
org.codehaus.janino janino 3.0.16
org.datanucleus datanucleus-api-jdo 4.2.4
org.datanucleus datanucleus-core 4.1.17
org.datanucleus datanucleus-rdbms 4.1.19
org.datanucleus javax.jdo 3.2.0-m3
org.eclipse.jetty jetty-client 9.4.43.v20210629
org.eclipse.jetty jetty-continuation 9.4.43.v20210629
org.eclipse.jetty jetty-http 9.4.43.v20210629
org.eclipse.jetty jetty-io 9.4.43.v20210629
org.eclipse.jetty jetty-jndi 9.4.43.v20210629
org.eclipse.jetty jetty-plus 9.4.43.v20210629
org.eclipse.jetty jetty-proxy 9.4.43.v20210629
org.eclipse.jetty jetty-security 9.4.43.v20210629
org.eclipse.jetty jetty-server 9.4.43.v20210629
org.eclipse.jetty jetty-servlet 9.4.43.v20210629
org.eclipse.jetty jetty-servlets 9.4.43.v20210629
org.eclipse.jetty jetty-util 9.4.43.v20210629
org.eclipse.jetty jetty-util-ajax 9.4.43.v20210629
org.eclipse.jetty jetty-webapp 9.4.43.v20210629
org.eclipse.jetty jetty-xml 9.4.43.v20210629
org.eclipse.jetty.websocket websocket-api 9.4.43.v20210629
org.eclipse.jetty.websocket websocket-client 9.4.43.v20210629
org.eclipse.jetty.websocket websocket-common 9.4.43.v20210629
org.eclipse.jetty.websocket websocket-server 9.4.43.v20210629
org.eclipse.jetty.websocket websocket-servlet 9.4.43.v20210629
org.fusesource.leveldbjni leveldbjni-all 1.8
org.glassfish.hk2 hk2-api 2.6.1
org.glassfish.hk2 hk2-locator 2.6.1
org.glassfish.hk2 hk2-utils 2.6.1
org.glassfish.hk2 osgi-resource-locator 1.0.3
org.glassfish.hk2.external aopalliance-repackaged 2.6.1
org.glassfish.hk2.external jakarta.inject 2.6.1
org.glassfish.jaxb jaxb-runtime 2.3.2
org.glassfish.jersey.containers jersey-container-servlet 2.34
org.glassfish.jersey.containers jersey-container-servlet-core 2.34
org.glassfish.jersey.core jersey-client 2.34
org.glassfish.jersey.core jersey-common 2.34
org.glassfish.jersey.core jersey-server 2.34
org.glassfish.jersey.inject jersey-hk2 2.34
org.hibernate.validator hibernate-validator 6.1.0.Final
org.javassist javassist 3.25.0-GA
org.jboss.logging jboss-logging 3.3.2.Final
org.jdbi jdbi 2.63.1
org.jetbrains annotations 17.0.0
org.joda joda-convert 1.7
org.jodd jodd-core 3.5.2
org.json4s json4s-ast_2.12 3.7.0-M11
org.json4s json4s-core_2.12 3.7.0-M11
org.json4s json4s-jackson_2.12 3.7.0-M11
org.json4s json4s-scalap_2.12 3.7.0-M11
org.lz4 lz4-java 1.7.1
org.mariadb.jdbc mariadb-java-client 2.2.5
org.objenesis objenesis 2.5.1
org.postgresql postgresql 42.2.19
org.roaringbitmap RoaringBitmap 0.9.14
org.roaringbitmap shims 0.9.14
org.rocksdb rocksdbjni 6.20.3
org.rosuda.REngine REngine 2.1.0
org.scala-lang scala-compiler_2.12 2.12.14
org.scala-lang scala-library_2.12 2.12.14
org.scala-lang scala-reflect_2.12 2.12.14
org.scala-lang.modules scala-collection-compat_2.12 2.4.3
org.scala-lang.modules scala-parser-combinators_2.12 1.1.2
org.scala-lang.modules scala-xml_2.12 1.2.0
org.scala-sbt test-interface 1.0
org.scalacheck scalacheck_2.12 1.14.2
org.scalactic scalactic_2.12 3.0.8
org.scalanlp breeze-macros_2.12 1.2
org.scalanlp breeze_2.12 1.2
org.scalatest scalatest_2.12 3.0.8
org.slf4j jcl-over-slf4j 1.7.30
org.slf4j jul-to-slf4j 1.7.30
org.slf4j slf4j-api 1.7.30
org.slf4j slf4j-log4j12 1.7.30
org.spark-project.spark unused 1.0.0
org.springframework spring-core 4.1.4.RELEASE
org.springframework spring-test 4.1.4.RELEASE
org.threeten threeten-extra 1.5.0
org.tukaani xz 1.8
org.typelevel algebra_2.12 2.0.1
org.typelevel cats-kernel_2.12 2.1.1
org.typelevel macro-compat_2.12 1.1.1
org.typelevel spire-macros_2.12 0.17.0
org.typelevel spire-platform_2.12 0.17.0
org.typelevel spire-util_2.12 0.17.0
org.typelevel spire_2.12 0.17.0
org.wildfly.openssl wildfly-openssl 1.0.7.Final
org.xerial sqlite-jdbc 3.8.11.2
org.xerial.snappy snappy-java 1.1.8.4
org.yaml snakeyaml 1.24
oro oro 2.0.8
pl.edu.icm JLargeArrays 1.5
software.amazon.ion ion-java 1.0.2
stax stax-api 1.0.1