Databricks Runtime 5.2 (unsupported)

  • Article

Databricks released this image in January 2019.

The following release notes provide information about Databricks Runtime 5.2, powered by Apache Spark.

New features

Databricks Advisor

Databricks Advisor automatically analyzes commands and displays advice notices to assist you in improving performance.

Databricks Advisor

See Databricks Advisor.

Delta Lake features

This Databricks Runtime version adds the following new Delta Lake features:

  • Fast Parquet import allows you to import Parquet files into a Delta table without copying data. This feature makes it easier to convert existing Parquet tables and migrate pipelines to Delta.

  • Time travel adds the ability to query a snapshot of a table using a timestamp string or a version, using SQL syntax as well as DataFrameReader options for timestamp expressions. This is a great tool for:

    • Re-creating analyses, reports, or outputs (for example, the output of a machine learning model), which is useful for debugging or auditing, especially in regulated fields.
    • Writing complex temporal queries.
    • Fixing mistakes in your data.
    • Providing snapshot isolation for a set of queries for fast changing tables.

    Important

    This feature is Experimental.

    See Work with Delta Lake table history.

Notebook-scoped libraries

Library utility (dbutils.library) (legacy) now support the Azure file system URI schemes adl, abfss, and wasbs. Library utilities let you install Python libraries, including Eggs, Wheels, and PyPI, into Python notebooks using the Databricks Utilities library API.

Improvements

  • Delta Lake
    • Table history now shows the snapshot version read and the isolation level used by a write to the Delta table.
    • Added support for ALTER TABLE SET LOCATION.
    • Added support for nested fields in UPDATE clause of MERGE INTO Delta table (for example, MERGE INTO table ON ... WHEN MATCHED THEN UPDATE SET topLevel.nestedField = ...).
  • Improved performance of approx_count_distinct aggregate function.
  • Upgraded Java from 1.8.0_181 to 1.8.0_191.
  • Upgraded Janino Java compiler from 3.0.9 to 3.0.10.
  • Removed Java library com.mchange.c3p0 and com.mchange.mchange-commons-java.
  • Upgraded setuptools in Python frmo 40.6.2 to 40.6.3.
  • Upgraded several installed R libraries. See Installed R Libraries.

Bug fixes

  • Make notebook display() respect Apache Spark session timezone config.

Apache Spark

Databricks Runtime 5.2 includes Apache Spark 2.4.0. This release includes all fixes and improvements included in Databricks Runtime 5.1 (unsupported), as well as the following additional bug fixes and improvements made to Spark:

  • [SPARK-26586][SS] Fix race condition that causes streams to run with unexpected confs
  • [SPARK-26267][SS] Retry when detecting incorrect offsets from Kafka (dbr-branch-5.x)
  • [SPARK-26383][CORE] NPE when use DataFrameReader.jdbc with wrong URL
  • [SPARK-26019][PYSPARK] Allow insecure py4j gateways
  • [SPARK-26098][WEBUI] Show associated SQL query in Job page
  • [SPARK-26402] Accessing nested fields with different cases in case insensitive mode
  • [SPARK-26444][WEBUI] Stage color doesn’t change with it’s status
  • [SPARK-26426][SQL] fix ExpresionInfo assert error in windows operation system.
  • [SPARK-26352][SQL] join reorder should not change the order of output attributes
  • [SPARK-26422][R] Support to disable Hive support in SparkR even for Hadoop versions unsupported by Hive fork
  • [SPARK-24687][CORE] Avoid job hanging when generate task binary causes fatal error
  • [SPARK-26366][SQL] ReplaceExceptWithFilter should consider NULL as False
  • [SPARK-26394][CORE] Fix annotation error for Utils.timeStringAsMs
  • [SPARK-26382][CORE] prefix comparator should handle -0.0
  • [SPARK-26352][SQL] Fix missing sameOutput in branch-2.4
  • [SPARK-26352][SQL] join reorder should not change the order of output attributes
  • [SPARK-26316][SPARK-21052] Revert hash join metrics in that causes performance degradation
  • [SPARK-26315][PYSPARK] auto cast threshold from Integer to Float in approxSimilarityJoin of BucketedRandomProjectionLSHModel
  • [SPARK-26265][CORE] Put freePage into a finally block
  • [SPARK-26370][SQL] Fix resolution of higher-order function for the same identifier.
  • [SPARK-21809]Change Stage Page to use datatables to support sorting columns and searching
  • [SPARK-26355][PYSPARK] Add a workaround for PyArrow 0.11.
  • [SPARK-26265][CORE] Fix deadlock in BytesToBytesMap.MapIterator when locking both BytesToBytesMap.MapIterator and TaskMemoryManager
  • [SPARK-26327][SQL] Bug fix for FileSourceScanExec metrics update
  • [SPARK-26307][SQL] Fix CTAS when INSERT a partitioned table using Hive serde
  • [SPARK-26233][SQL] CheckOverflow when encoding a decimal value
  • [SPARK-26119][CORE][WEBUI] Task summary table should contain only successful tasks’ metrics
  • [SPARK-24423][SQL] Fix error example
  • [SPARK-25860][SQL] Replace Literal(null, _) with FalseLiteral whenever possible
  • [SPARK-26219][CORE] Executor summary should get updated for failure jobs in the history server UI
  • [SPARK-26226][SQL] Update query tracker to report timeline for phases
  • [SPARK-26181][SQL] the hasMinMaxStats method of ColumnStatsMap is not correct
  • [SPARK-26198][SQL] Fix Metadata serialize null values throw NPE
  • [SPARK-26080][PYTHON] Skips Python resource limit on Windows in Python worker
  • [SPARK-26241][SQL] Add queryId to IncrementalExecution (backport)
  • [SPARK-26201] Fix python broadcast with encryption
  • [SPARK-26188][SQL] FileIndex: don’t infer data types of partition columns if user specifies schema
  • [SPARK-26186][SPARK-26184][CORE] Last updated time is not getting updated for the Inprogress application
  • [SPARK-24341][SQL] Support only IN subqueries with the same number of items per row
  • [SPARK-26137][CORE] Use Java system property “file.separator” inste…
  • [SPARK-26114][CORE] ExternalSorter’s readingIterator field leak
  • [SPARK-25451][SPARK-26100][CORE] Aggregated metrics table doesn’t show the right number of the total tasks
  • [SPARK-25786][CORE] If the ByteBuffer.hasArray is false , it will throw UnsupportedOperationException for Kryo
  • [SPARK-24553][UI] Fix unnecessary UI redirect
  • [SPARK-26109][WEBUI] Duration in the task summary metrics table and the task table are different
  • [SPARK-26084][SQL] Fixes unresolved AggregateExpression.references exception
  • [SPARK-25947][SQL][BACKPORT-5.X] Reduce memory usage in ShuffleExchangeExec by selecting only the sort columns
  • [SPARK-26118][WEB UI] Introducing spark.ui.requestHeaderSize for setting HTTP requestHeaderSize
  • [SPARK-26079][SQL] Ensure listener event delivery in StreamingQueryListenersConfSuite.
  • [SPARK-26092][SS] Use CheckpointFileManager to write the streaming metadata file
  • [SPARK-25883][SQL] Override method prettyName in from_avro/to_avro

Maintenance updates

See Databricks Runtime 5.2 maintenance updates.

System environment

  • Operating System: Ubuntu 16.04.5 LTS
  • Java: 1.8.0_191
  • Scala: 2.11.8
  • Python: 2.7.12 for Python 2 clusters and 3.5.2 for Python 3 clusters.
  • R: R version 3.4.4 (2018-03-15)
  • GPU clusters: The following NVIDIA GPU libraries are installed:
    • Tesla driver 375.66
    • CUDA 9.0
    • cuDNN 7.0

Note

Although Scala 2.12 is supported in Apache Spark 2.4, it is not supported in Databricks Runtime 5.2.

Installed Python libraries

Library Version Library Version Library Version
ansi2html 1.1.1 argparse 1.2.1 backports-abc 0.5
boto 2.42.0 boto3 1.4.1 botocore 1.4.70
brewer2mpl 1.4.1 certifi 2016.2.28 cffi 1.7.0
chardet 2.3.0 colorama 0.3.7 configobj 5.0.6
cryptography 1.5 cycler 0.10.0 Cython 0.24.1
decorator 4.0.10 docutils 0.14 enum34 1.1.6
et-xmlfile 1.0.1 freetype-py 1.0.2 funcsigs 1.0.2
fusepy 2.0.4 futures 3.2.0 ggplot 0.6.8
html5lib 0.999 idna 2.1 ipaddress 1.0.16
ipython 2.2.0 ipython-genutils 0.1.0 jdcal 1.2
Jinja2 2.8 jmespath 0.9.0 llvmlite 0.13.0
lxml 3.6.4 MarkupSafe 0.23 matplotlib 1.5.3
mpld3 0.2 msgpack-python 0.4.7 ndg-httpsclient 0.3.3
numba 0.28.1 numpy 1.11.1 openpyxl 2.3.2
pandas 0.19.2 pathlib2 2.1.0 patsy 0.4.1
pexpect 4.0.1 pickleshare 0.7.4 Pillow 3.3.1
pip 18.1 ply 3.9 prompt-toolkit 1.0.7
psycopg2 2.6.2 ptyprocess 0.5.1 py4j 0.10.3
pyarrow 0.8.0 pyasn1 0.1.9 pycparser 2.14
Pygments 2.1.3 PyGObject 3.20.0 pyOpenSSL 16.0.0
pyparsing 2.2.0 pypng 0.0.18 Python 2.7.12
python-dateutil 2.5.3 python-geohash 0.8.5 pytz 2016.6.1
requests 2.11.1 s3transfer 0.1.9 scikit-learn 0.18.1
scipy 0.18.1 scour 0.32 seaborn 0.7.1
setuptools 40.6.3 simplejson 3.8.2 simples3 1.0
singledispatch 3.4.0.3 six 1.10.0 statsmodels 0.6.1
tornado 5.1.1 traitlets 4.3.0 urllib3 1.19.1
virtualenv 16.1.0 wcwidth 0.1.7 wheel 0.32.3
wsgiref 0.1.2

Installed R libraries

Library Version Library Version Library Version
abind 1.4-5 assertthat 0.2.0 backports 1.1.3
base 3.4.4 BH 1.66.0-1 bindr 0.1.1
bindrcpp 0.2.2 bit 1.1-14 bit64 0.9-7
bitops 1.0-6 blob 1.1.1 boot 1.3-20
brew 1.0-6 callr 3.1.1 car 3.0-2
carData 3.0-2 caret 6.0-81 cellranger 1.1.0
chron 2.3-53 class 7.3-14 cli 1.0.1
clipr 0.4.1 clisymbols 1.2.0 cluster 2.0.7-1
codetools 0.2-15 colorspace 1.3-2 commonmark 1.7
compiler 3.4.4 crayon 1.3.4 curl 3.2
data.table 1.11.8 datasets 3.4.4 DBI 1.0.0
desc 1.2.0 devtools 2.0.1 digest 0.6.18
doMC 1.3.5 dplyr 0.7.8 fansi 0.4.0
forcats 0.3.0 foreach 1.4.4 foreign 0.8-70
fs 1.2.6 gbm 2.1.4 generics 0.0.2
ggplot2 3.1.0 gh 1.0.1 git2r 0.23.0
glmnet 2.0-16 glue 1.3.0 gower 0.1.2
graphics 3.4.4 grDevices 3.4.4 grid 3.4.4
gridExtra 2.3 gsubfn 0.7 gtable 0.2.0
h2o 3.20.0.8 haven 2.0.0 hms 0.4.2
httr 1.4.0 hwriter 1.3.2 hwriterPlus 1.0-3
ini 0.3.1 ipred 0.9-8 iterators 1.0.10
jsonlite 1.6 KernSmooth 2.23-15 labeling 0.3
lattice 0.20-38 lava 1.6.4 lazyeval 0.2.1
littler 0.3.5 lme4 1.1-19 lubridate 1.7.4
magrittr 1.5 mapproj 1.2.6 maps 3.3.0
maptools 0.9-4 MASS 7.3-50 Matrix 1.2-14
MatrixModels 0.4-1 memoise 1.1.0 methods 3.4.4
mgcv 1.8-25 mime 0.6 minqa 1.2.4
ModelMetrics 1.2.2 munsell 0.5.0 mvtnorm 1.0-8
nlme 3.1-137 nloptr 1.2.1 nnet 7.3-12
numDeriv 2016.8-1 openssl 1.1 openxlsx 4.1.0
parallel 3.4.4 pbkrtest 0.4-7 pillar 1.3.1
pkgbuild 1.0.2 pkgconfig 2.0.2 pkgKitten 0.1.4
pkgload 1.0.2 plogr 0.2.0 plyr 1.8.4
praise 1.0.0 prettyunits 1.0.2 pROC 1.13.0
processx 3.2.1 prodlim 2018.04.18 progress 1.2.0
proto 1.0.0 ps 1.3.0 purrr 0.2.5
quantreg 5.38 R.methodsS3 1.7.1 R.oo 1.22.0
R.utils 2.7.0 R6 2.3.0 randomForest 4.6-14
rcmdcheck 1.3.2 RColorBrewer 1.1-2 Rcpp 1.0.0
RcppEigen 0.3.3.5.0 RcppRoll 0.3.0 RCurl 1.95-4.11
readr 1.3.1 readxl 1.2.0 recipes 0.1.4
rematch 1.0.1 remotes 2.0.2 reshape2 1.4.3
rio 0.5.16 rlang 0.3.0.1 RODBC 1.3-15
roxygen2 6.1.1 rpart 4.1-13 rprojroot 1.3-2
Rserve 1.7-3 RSQLite 2.1.1 rstudioapi 0.8
scales 1.0.0 sessioninfo 1.1.1 sp 1.3-1
SparkR 2.4.0 SparseM 1.77 spatial 7.3-11
splines 3.4.4 sqldf 0.4-11 SQUAREM 2017.10-1
statmod 1.4.30 stats 3.4.4 stats4 3.4.4
stringi 1.2.4 stringr 1.3.1 survival 2.43-3
tcltk 3.4.4 TeachingDemos 2.10 testthat 2.0.1
tibble 1.4.2 tidyr 0.8.2 tidyselect 0.2.5
timeDate 3043.102 tools 3.4.4 usethis 1.4.0
utf8 1.1.4 utils 3.4.4 viridisLite 0.3.0
whisker 0.3-2 withr 2.1.2 xml2 1.2.0
xopen 1.0.0 zip 1.0.0

Installed Java and Scala libraries (Scala 2.11 cluster version)

Group ID Artifact ID Version
antlr antlr 2.7.7
com.amazonaws amazon-kinesis-client 1.8.10
com.amazonaws aws-java-sdk-autoscaling 1.11.313
com.amazonaws aws-java-sdk-cloudformation 1.11.313
com.amazonaws aws-java-sdk-cloudfront 1.11.313
com.amazonaws aws-java-sdk-cloudhsm 1.11.313
com.amazonaws aws-java-sdk-cloudsearch 1.11.313
com.amazonaws aws-java-sdk-cloudtrail 1.11.313
com.amazonaws aws-java-sdk-cloudwatch 1.11.313
com.amazonaws aws-java-sdk-cloudwatchmetrics 1.11.313
com.amazonaws aws-java-sdk-codedeploy 1.11.313
com.amazonaws aws-java-sdk-cognitoidentity 1.11.313
com.amazonaws aws-java-sdk-cognitosync 1.11.313
com.amazonaws aws-java-sdk-config 1.11.313
com.amazonaws aws-java-sdk-core 1.11.313
com.amazonaws aws-java-sdk-datapipeline 1.11.313
com.amazonaws aws-java-sdk-directconnect 1.11.313
com.amazonaws aws-java-sdk-directory 1.11.313
com.amazonaws aws-java-sdk-dynamodb 1.11.313
com.amazonaws aws-java-sdk-ec2 1.11.313
com.amazonaws aws-java-sdk-ecs 1.11.313
com.amazonaws aws-java-sdk-efs 1.11.313
com.amazonaws aws-java-sdk-elasticache 1.11.313
com.amazonaws aws-java-sdk-elasticbeanstalk 1.11.313
com.amazonaws aws-java-sdk-elasticloadbalancing 1.11.313
com.amazonaws aws-java-sdk-elastictranscoder 1.11.313
com.amazonaws aws-java-sdk-emr 1.11.313
com.amazonaws aws-java-sdk-glacier 1.11.313
com.amazonaws aws-java-sdk-iam 1.11.313
com.amazonaws aws-java-sdk-importexport 1.11.313
com.amazonaws aws-java-sdk-kinesis 1.11.313
com.amazonaws aws-java-sdk-kms 1.11.313
com.amazonaws aws-java-sdk-lambda 1.11.313
com.amazonaws aws-java-sdk-logs 1.11.313
com.amazonaws aws-java-sdk-machinelearning 1.11.313
com.amazonaws aws-java-sdk-opsworks 1.11.313
com.amazonaws aws-java-sdk-rds 1.11.313
com.amazonaws aws-java-sdk-redshift 1.11.313
com.amazonaws aws-java-sdk-route53 1.11.313
com.amazonaws aws-java-sdk-s3 1.11.313
com.amazonaws aws-java-sdk-ses 1.11.313
com.amazonaws aws-java-sdk-simpledb 1.11.313
com.amazonaws aws-java-sdk-simpleworkflow 1.11.313
com.amazonaws aws-java-sdk-sns 1.11.313
com.amazonaws aws-java-sdk-sqs 1.11.313
com.amazonaws aws-java-sdk-ssm 1.11.313
com.amazonaws aws-java-sdk-storagegateway 1.11.313
com.amazonaws aws-java-sdk-sts 1.11.313
com.amazonaws aws-java-sdk-support 1.11.313
com.amazonaws aws-java-sdk-swf-libraries 1.11.22
com.amazonaws aws-java-sdk-workspaces 1.11.313
com.amazonaws jmespath-java 1.11.313
com.carrotsearch hppc 0.7.2
com.chuusai shapeless_2.11 2.3.2
com.clearspring.analytics stream 2.7.0
com.databricks Rserve 1.8-3
com.databricks dbml-local_2.11 0.5.0-db8-spark2.4
com.databricks dbml-local_2.11-tests 0.5.0-db8-spark2.4
com.databricks jets3t 0.7.1-0
com.databricks.scalapb compilerplugin_2.11 0.4.15-9
com.databricks.scalapb scalapb-runtime_2.11 0.4.15-9
com.esotericsoftware kryo-shaded 4.0.2
com.esotericsoftware minlog 1.3.0
com.fasterxml classmate 1.0.0
com.fasterxml.jackson.core jackson-annotations 2.6.7
com.fasterxml.jackson.core jackson-core 2.6.7
com.fasterxml.jackson.core jackson-databind 2.6.7.1
com.fasterxml.jackson.dataformat jackson-dataformat-cbor 2.6.7
com.fasterxml.jackson.datatype jackson-datatype-joda 2.6.7
com.fasterxml.jackson.module jackson-module-paranamer 2.6.7
com.fasterxml.jackson.module jackson-module-scala_2.11 2.6.7.1
com.github.fommil jniloader 1.1
com.github.fommil.netlib core 1.1.2
com.github.fommil.netlib native_ref-java 1.1
com.github.fommil.netlib native_ref-java-natives 1.1
com.github.fommil.netlib native_system-java 1.1
com.github.fommil.netlib native_system-java-natives 1.1
com.github.fommil.netlib netlib-native_ref-linux-x86_64-natives 1.1
com.github.fommil.netlib netlib-native_system-linux-x86_64-natives 1.1
com.github.luben zstd-jni 1.3.2-2
com.github.rwl jtransforms 2.4.0
com.google.code.findbugs jsr305 2.0.1
com.google.code.gson gson 2.2.4
com.google.guava guava 15.0
com.google.protobuf protobuf-java 2.6.1
com.googlecode.javaewah JavaEWAH 0.3.2
com.h2database h2 1.3.174
com.jcraft jsch 0.1.50
com.jolbox bonecp 0.8.0.RELEASE
com.microsoft.azure azure-data-lake-store-sdk 2.2.8
com.microsoft.sqlserver mssql-jdbc 6.2.2.jre8
com.ning compress-lzf 1.0.3
com.sun.mail javax.mail 1.5.2
com.thoughtworks.paranamer paranamer 2.8
com.trueaccord.lenses lenses_2.11 0.3
com.twitter chill-java 0.9.3
com.twitter chill_2.11 0.9.3
com.twitter parquet-hadoop-bundle 1.6.0
com.twitter util-app_2.11 6.23.0
com.twitter util-core_2.11 6.23.0
com.twitter util-jvm_2.11 6.23.0
com.typesafe config 1.2.1
com.typesafe.scala-logging scala-logging-api_2.11 2.1.2
com.typesafe.scala-logging scala-logging-slf4j_2.11 2.1.2
com.univocity univocity-parsers 2.7.3
com.vlkan flatbuffers 1.2.0-3f79e055
com.zaxxer HikariCP 3.1.0
commons-beanutils commons-beanutils 1.7.0
commons-beanutils commons-beanutils-core 1.8.0
commons-cli commons-cli 1.2
commons-codec commons-codec 1.10
commons-collections commons-collections 3.2.2
commons-configuration commons-configuration 1.6
commons-dbcp commons-dbcp 1.4
commons-digester commons-digester 1.8
commons-httpclient commons-httpclient 3.1
commons-io commons-io 2.4
commons-lang commons-lang 2.6
commons-logging commons-logging 1.1.3
commons-net commons-net 3.1
commons-pool commons-pool 1.5.4
info.ganglia.gmetric4j gmetric4j 1.0.7
io.airlift aircompressor 0.10
io.dropwizard.metrics metrics-core 3.1.5
io.dropwizard.metrics metrics-ganglia 3.1.5
io.dropwizard.metrics metrics-graphite 3.1.5
io.dropwizard.metrics metrics-healthchecks 3.1.5
io.dropwizard.metrics metrics-jetty9 3.1.5
io.dropwizard.metrics metrics-json 3.1.5
io.dropwizard.metrics metrics-jvm 3.1.5
io.dropwizard.metrics metrics-log4j 3.1.5
io.dropwizard.metrics metrics-servlets 3.1.5
io.netty netty 3.9.9.Final
io.netty netty-all 4.1.17.Final
io.prometheus simpleclient 0.0.16
io.prometheus simpleclient_common 0.0.16
io.prometheus simpleclient_dropwizard 0.0.16
io.prometheus simpleclient_servlet 0.0.16
io.prometheus.jmx collector 0.7
javax.activation activation 1.1.1
javax.annotation javax.annotation-api 1.2
javax.el javax.el-api 2.2.4
javax.jdo jdo-api 3.0.1
javax.servlet javax.servlet-api 3.1.0
javax.servlet.jsp jsp-api 2.1
javax.transaction jta 1.1
javax.validation validation-api 1.1.0.Final
javax.ws.rs javax.ws.rs-api 2.0.1
javax.xml.bind jaxb-api 2.2.2
javax.xml.stream stax-api 1.0-2
javolution javolution 5.5.1
jline jline 2.11
joda-time joda-time 2.9.3
log4j apache-log4j-extras 1.2.17
log4j log4j 1.2.17
net.hydromatic eigenbase-properties 1.1.5
net.razorvine pyrolite 4.13
net.sf.jpam jpam 1.1
net.sf.opencsv opencsv 2.3
net.sf.supercsv super-csv 2.2.0
net.snowflake snowflake-ingest-sdk 0.9.5
net.snowflake snowflake-jdbc 3.6.15
net.snowflake spark-snowflake_2.11 2.4.10-spark_2.4
net.sourceforge.f2j arpack_combined_all 0.1
org.acplt oncrpc 1.0.7
org.antlr ST4 4.0.4
org.antlr antlr-runtime 3.4
org.antlr antlr4-runtime 4.7
org.antlr stringtemplate 3.2.1
org.apache.ant ant 1.9.2
org.apache.ant ant-jsch 1.9.2
org.apache.ant ant-launcher 1.9.2
org.apache.arrow arrow-format 0.10.0
org.apache.arrow arrow-memory 0.10.0
org.apache.arrow arrow-vector 0.10.0
org.apache.avro avro 1.8.2
org.apache.avro avro-ipc 1.8.2
org.apache.avro avro-mapred-hadoop2 1.8.2
org.apache.calcite calcite-avatica 1.2.0-incubating
org.apache.calcite calcite-core 1.2.0-incubating
org.apache.calcite calcite-linq4j 1.2.0-incubating
org.apache.commons commons-compress 1.8.1
org.apache.commons commons-crypto 1.0.0
org.apache.commons commons-lang3 3.5
org.apache.commons commons-math3 3.4.1
org.apache.curator curator-client 2.7.1
org.apache.curator curator-framework 2.7.1
org.apache.curator curator-recipes 2.7.1
org.apache.derby derby 10.12.1.1
org.apache.directory.api api-asn1-api 1.0.0-M20
org.apache.directory.api api-util 1.0.0-M20
org.apache.directory.server apacheds-i18n 2.0.0-M15
org.apache.directory.server apacheds-kerberos-codec 2.0.0-M15
org.apache.hadoop hadoop-annotations 2.7.3
org.apache.hadoop hadoop-auth 2.7.3
org.apache.hadoop hadoop-client 2.7.3
org.apache.hadoop hadoop-common 2.7.3
org.apache.hadoop hadoop-hdfs 2.7.3
org.apache.hadoop hadoop-mapreduce-client-app 2.7.3
org.apache.hadoop hadoop-mapreduce-client-common 2.7.3
org.apache.hadoop hadoop-mapreduce-client-core 2.7.3
org.apache.hadoop hadoop-mapreduce-client-jobclient 2.7.3
org.apache.hadoop hadoop-mapreduce-client-shuffle 2.7.3
org.apache.hadoop hadoop-yarn-api 2.7.3
org.apache.hadoop hadoop-yarn-client 2.7.3
org.apache.hadoop hadoop-yarn-common 2.7.3
org.apache.hadoop hadoop-yarn-server-common 2.7.3
org.apache.htrace htrace-core 3.1.0-incubating
org.apache.httpcomponents httpclient 4.5.4
org.apache.httpcomponents httpcore 4.4.8
org.apache.ivy ivy 2.4.0
org.apache.orc orc-core-nohive 1.5.2
org.apache.orc orc-mapreduce-nohive 1.5.2
org.apache.orc orc-shims 1.5.2
org.apache.parquet parquet-column 1.10.1-databricks3
org.apache.parquet parquet-common 1.10.1-databricks3
org.apache.parquet parquet-encoding 1.10.1-databricks3
org.apache.parquet parquet-format 2.4.0
org.apache.parquet parquet-hadoop 1.10.1-databricks3
org.apache.parquet parquet-jackson 1.10.1-databricks3
org.apache.thrift libfb303 0.9.3
org.apache.thrift libthrift 0.9.3
org.apache.xbean xbean-asm6-shaded 4.8
org.apache.zookeeper zookeeper 3.4.6
org.codehaus.jackson jackson-core-asl 1.9.13
org.codehaus.jackson jackson-jaxrs 1.9.13
org.codehaus.jackson jackson-mapper-asl 1.9.13
org.codehaus.jackson jackson-xc 1.9.13
org.codehaus.janino commons-compiler 3.0.10
org.codehaus.janino janino 3.0.10
org.datanucleus datanucleus-api-jdo 3.2.6
org.datanucleus datanucleus-core 3.2.10
org.datanucleus datanucleus-rdbms 3.2.9
org.eclipse.jetty jetty-client 9.3.20.v20170531
org.eclipse.jetty jetty-continuation 9.3.20.v20170531
org.eclipse.jetty jetty-http 9.3.20.v20170531
org.eclipse.jetty jetty-io 9.3.20.v20170531
org.eclipse.jetty jetty-jndi 9.3.20.v20170531
org.eclipse.jetty jetty-plus 9.3.20.v20170531
org.eclipse.jetty jetty-proxy 9.3.20.v20170531
org.eclipse.jetty jetty-security 9.3.20.v20170531
org.eclipse.jetty jetty-server 9.3.20.v20170531
org.eclipse.jetty jetty-servlet 9.3.20.v20170531
org.eclipse.jetty jetty-servlets 9.3.20.v20170531
org.eclipse.jetty jetty-util 9.3.20.v20170531
org.eclipse.jetty jetty-webapp 9.3.20.v20170531
org.eclipse.jetty jetty-xml 9.3.20.v20170531
org.fusesource.leveldbjni leveldbjni-all 1.8
org.glassfish.hk2 hk2-api 2.4.0-b34
org.glassfish.hk2 hk2-locator 2.4.0-b34
org.glassfish.hk2 hk2-utils 2.4.0-b34
org.glassfish.hk2 osgi-resource-locator 1.0.1
org.glassfish.hk2.external aopalliance-repackaged 2.4.0-b34
org.glassfish.hk2.external javax.inject 2.4.0-b34
org.glassfish.jersey.bundles.repackaged jersey-guava 2.22.2
org.glassfish.jersey.containers jersey-container-servlet 2.22.2
org.glassfish.jersey.containers jersey-container-servlet-core 2.22.2
org.glassfish.jersey.core jersey-client 2.22.2
org.glassfish.jersey.core jersey-common 2.22.2
org.glassfish.jersey.core jersey-server 2.22.2
org.glassfish.jersey.media jersey-media-jaxb 2.22.2
org.hibernate hibernate-validator 5.1.1.Final
org.iq80.snappy snappy 0.2
org.javassist javassist 3.18.1-GA
org.jboss.logging jboss-logging 3.1.3.GA
org.jdbi jdbi 2.63.1
org.joda joda-convert 1.7
org.jodd jodd-core 3.5.2
org.json4s json4s-ast_2.11 3.5.3
org.json4s json4s-core_2.11 3.5.3
org.json4s json4s-jackson_2.11 3.5.3
org.json4s json4s-scalap_2.11 3.5.3
org.lz4 lz4-java 1.4.0
org.mariadb.jdbc mariadb-java-client 2.1.2
org.mockito mockito-all 1.9.5
org.objenesis objenesis 2.5.1
org.postgresql postgresql 42.1.4
org.roaringbitmap RoaringBitmap 0.5.11
org.rocksdb rocksdbjni 5.2.1
org.rosuda.REngine REngine 2.1.0
org.scala-lang scala-compiler_2.11 2.11.8
org.scala-lang scala-library_2.11 2.11.8
org.scala-lang scala-reflect_2.11 2.11.8
org.scala-lang.modules scala-parser-combinators_2.11 1.0.2
org.scala-lang.modules scala-xml_2.11 1.0.5
org.scala-sbt test-interface 1.0
org.scalacheck scalacheck_2.11 1.12.5
org.scalactic scalactic_2.11 3.0.3
org.scalanlp breeze-macros_2.11 0.13.2
org.scalanlp breeze_2.11 0.13.2
org.scalatest scalatest_2.11 3.0.3
org.slf4j jcl-over-slf4j 1.7.16
org.slf4j jul-to-slf4j 1.7.16
org.slf4j slf4j-api 1.7.16
org.slf4j slf4j-log4j12 1.7.16
org.spark-project.hive hive-beeline 1.2.1.spark2
org.spark-project.hive hive-cli 1.2.1.spark2
org.spark-project.hive hive-exec 1.2.1.spark2
org.spark-project.hive hive-jdbc 1.2.1.spark2
org.spark-project.hive hive-metastore 1.2.1.spark2
org.spark-project.spark unused 1.0.0
org.spire-math spire-macros_2.11 0.13.0
org.spire-math spire_2.11 0.13.0
org.springframework spring-core 4.1.4.RELEASE
org.springframework spring-test 4.1.4.RELEASE
org.tukaani xz 1.5
org.typelevel machinist_2.11 0.6.1
org.typelevel macro-compat_2.11 1.1.1
org.xerial sqlite-jdbc 3.8.11.2
org.xerial.snappy snappy-java 1.1.7.1
org.yaml snakeyaml 1.16
oro oro 2.0.8
software.amazon.ion ion-java 1.0.2
stax stax-api 1.0.1
xmlenc xmlenc 0.52