Databricks Runtime 8.1 för ML (stöds inte)
Databricks släppte den här bilden i mars 2021.
Databricks Runtime 8.1 för Machine Learning ger en färdig miljö för maskininlärning och datavetenskap baserat på Databricks Runtime 8.1 (stöds inte). Databricks Runtime ML innehåller många populära maskininlärningsbibliotek, inklusive TensorFlow, PyTorch och XGBoost. Den stöder även distribuerad djupinlärningsträning med Horovod.
Mer information, inklusive instruktioner för att skapa ett Databricks Runtime ML-kluster, finns i AI och Machine Learning på Databricks.
Nya funktioner och större ändringar
Databricks Runtime 8.1 ML bygger på Databricks Runtime 8.1. Information om nyheter i Databricks Runtime 8.1, inklusive Apache Spark MLlib och SparkR, finns i viktig information om Databricks Runtime 8.1 (stöds inte).
Paket som tagits bort i GPU-kluster
Följande CUDA-paket tas bort i GPU-klustren:
- cuda-command-line-tools
- cuda-compiler
- cuda-cudart-dev
- cuda-manschett
- cuda-cufft-dev
- cuda-cuobjdump
- cuda-cupti
- cuda-curand
- cuda-curand-dev
- cuda-cusolver
- cuda-cusolver-dev
- cuda-cusparse
- cuda-cusparse-dev
- cuda-documentation
- cuda-driver-dev
- cuda-gdb
- cuda-gpu-library-advisor
- cuda-libraries-dev
- cuda-license
- cuda-memcheck
- cuda-minimal-build
- cuda-misc-headers
- cuda-npp
- cuda-npp-dev
- cuda-nsight
- cuda-nvcc
- cuda-nvdisasm
- cuda-nvgraph
- cuda-nvgraph-dev
- cuda-nvjpeg
- cuda-nvjpeg-dev
- cuda-nvml-dev
- cuda-nvprune
- cuda-nvrtc-dev
- cuda-nvvp
- cuda-samples
- cuda-sanitizer-api
- cuda-toolkit
- cuda-tools
- cuda-visual-tools
- freeglut3
- libcublas-dev
- libcudnn7-dev
- libdrm-dev
- libegl1
- libegm-mesa0
- libgbl1-mesa-dev
- libgbm1
- libgles1
- libgles2
- libglu1-mesa
- libglu1-mesa-dev
- libnccl-dev
- libnvinfer-dev
- libnvinfer-plugin-dev
- libopengl0
- libwayland-server0
- libx11-xcb-dev
- libxcb-dri2-0-dev
- libxcb-dri3-dev
- libxcb-glx0-dev
- libxcb-present-dev
- libxcb-randr0
- libxcb-randr0-dev
- libxcb-render0-dev
- libxcb-shape0-dev
- libxcb-sync-dev
- libxcb-xfixes0
- libxcb-xfixes0-dev
- libxdamage-dev
- libxext-dev
- libxfixes-dev
- libxi-dev
- libxmu-dev
- libxmu-headers
- libxshmfence-dev
- libxxf86vm-dev
- mesa-common-dev
- nsight-compute
- nsight-system
- x11proto-damage-dev
- x11proto-fixes-dev
- x11proto-input-dev
- x11proto-xext-dev
- x11proto-xf86vidmode-dev
Större ändringar i Databricks Runtime ML Python-miljön
Se Databricks Runtime 8.1 (stöds inte) för större ändringar i Databricks Runtime Python-miljön. En fullständig lista över installerade Python-paket och deras versioner finns i Python-bibliotek.
Uppgraderade Python-paket
- mlflow 1.13.1 -> 1.14.1
- plotly 4.14.1 -> 4.14.3
- pytz 2020.1 -> 2020.5
- shap 0.37.0 -> 0.38.1
- tensorflow 2.4.0 -> 2.4.1
- torchvision 0.8.1 -> 0.8.2
- xgboost 1.3.1 -> 1.3.3
Systemmiljö
Systemmiljön i Databricks Runtime 8.1 ML skiljer sig från Databricks Runtime 8.1 på följande sätt:
- DBUtils: Databricks Runtime ML innehåller inte biblioteksverktyget (dbutils.library) (äldre).
Använd
%pip
och%conda
kommandon i stället. Se Python-bibliotek med notebook-omfång. - För GPU-kluster innehåller Databricks Runtime ML följande NVIDIA GPU-bibliotek:
- CUDA 11.0
- cuDNN 8.0.4.30
- NCCL 2.7.8
- TensorRT 7.1.3
Bibliotek
I följande avsnitt visas de bibliotek som ingår i Databricks Runtime 8.1 ML som skiljer sig från de som ingår i Databricks Runtime 8.1.
I detta avsnitt:
- Bibliotek på den översta nivån
- Python-bibliotek
- R-bibliotek
- Java- och Scala-bibliotek (Scala 2.12-kluster)
Bibliotek på den översta nivån
Databricks Runtime 8.1 ML innehåller följande bibliotek på den översta nivån:
- GraphFrames
- Horovod och HorovodRunner
- MLflow
- PyTorch
- spark-tensorflow-connector
- TensorFlow
- TensorBoard
Python-bibliotek
Databricks Runtime 8.1 ML använder Conda för Python-pakethantering och innehåller många populära ML-paket.
Förutom de paket som anges i Conda-miljöerna i följande avsnitt innehåller Databricks Runtime 8.1 ML även följande paket:
- hyperopt 0.2.5.db1
- sparkdl 2.1.0.db4
Python-bibliotek i CPU-kluster
name: databricks-ml
channels:
- pytorch
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- absl-py=0.11.0=pyhd3eb1b0_1
- aiohttp=3.7.4=py38h27cfd23_1
- asn1crypto=1.4.0=py_0
- astor=0.8.1=py38h06a4308_0
- async-timeout=3.0.1=py38h06a4308_0
- attrs=20.3.0=pyhd3eb1b0_0
- backcall=0.2.0=pyhd3eb1b0_0
- bcrypt=3.2.0=py38h7b6447c_0
- blas=1.0=mkl
- blinker=1.4=py38h06a4308_0
- boto3=1.16.7=pyhd3eb1b0_0
- botocore=1.19.7=pyhd3eb1b0_0
- brotlipy=0.7.0=py38h27cfd23_1003
- c-ares=1.17.1=h27cfd23_0
- ca-certificates=2021.4.13=h06a4308_1 # (updated from 2021.1.19 in May 26, 2021 maintenance update)
- cachetools=4.2.1=pyhd3eb1b0_0
- certifi=2020.12.5=py38h06a4308_0
- cffi=1.14.3=py38h261ae71_2
- chardet=3.0.4=py38h06a4308_1003
- click=7.1.2=pyhd3eb1b0_0
- cloudpickle=1.6.0=py_0
- configparser=5.0.1=py_0
- cpuonly=1.0=0
- cryptography=3.1.1=py38h1ba5d50_0
- cycler=0.10.0=py38_0
- cython=0.29.21=py38h2531618_0
- decorator=4.4.2=pyhd3eb1b0_0
- dill=0.3.2=py_0
- docutils=0.15.2=py38h06a4308_1
- entrypoints=0.3=py38_0
- flask=1.1.2=pyhd3eb1b0_0
- freetype=2.10.4=h5ab3b9f_0
- future=0.18.2=py38_1
- gitdb=4.0.5=py_0
- gitpython=3.1.12=pyhd3eb1b0_1
- google-auth=1.22.1=py_0
- google-auth-oauthlib=0.4.2=pyhd3eb1b0_2
- google-pasta=0.2.0=py_0
- gunicorn=20.0.4=py38_0
- h5py=2.10.0=py38h7918eee_0
- hdf5=1.10.4=hb1b8bf9_0
- icu=58.2=he6710b0_3
- idna=2.10=pyhd3eb1b0_0
- importlib-metadata=2.0.0=py_1
- intel-openmp=2019.4=243
- ipykernel=5.3.4=py38h5ca1d4c_0
- ipython=7.19.0=py38hb070fc8_1
- ipython_genutils=0.2.0=pyhd3eb1b0_1
- isodate=0.6.0=py_1
- itsdangerous=1.1.0=pyhd3eb1b0_0
- jedi=0.17.2=py38h06a4308_1
- jinja2=2.11.2=pyhd3eb1b0_0
- jmespath=0.10.0=py_0
- joblib=0.17.0=py_0
- jpeg=9b=h024ee3a_2
- jupyter_client=6.1.7=py_0
- jupyter_core=4.6.3=py38_0
- kiwisolver=1.3.0=py38h2531618_0
- krb5=1.17.1=h173b8e3_0
- lcms2=2.11=h396b838_0
- ld_impl_linux-64=2.33.1=h53a641e_7
- libedit=3.1.20191231=h14c3975_1
- libffi=3.3=he6710b0_2
- libgcc-ng=9.1.0=hdf63c60_0
- libgfortran-ng=7.3.0=hdf63c60_0
- libpng=1.6.37=hbc83047_0
- libpq=12.2=h20c2e04_0
- libprotobuf=3.13.0.1=hd408876_0
- libsodium=1.0.18=h7b6447c_0
- libstdcxx-ng=9.1.0=hdf63c60_0
- libtiff=4.1.0=h2733197_1
- libuv=1.40.0=h7b6447c_0
- lightgbm=3.1.1=py38h2531618_0
- lz4-c=1.9.2=heb0550a_3
- mako=1.1.3=py_0
- markdown=3.3.3=py38h06a4308_0
- markupsafe=1.1.1=py38h7b6447c_0
- matplotlib-base=3.2.2=py38hef1b27d_0
- mkl=2019.4=243
- mkl-service=2.3.0=py38he904b0f_0
- mkl_fft=1.2.0=py38h23d657b_0
- mkl_random=1.1.0=py38h962f231_0
- more-itertools=8.6.0=pyhd3eb1b0_0
- multidict=5.1.0=py38h27cfd23_2
- ncurses=6.2=he6710b0_1
- networkx=2.5=py_0
- ninja=1.10.2=py38hff7bd54_0
- nltk=3.5=py_0
- numpy=1.19.2=py38h54aff64_0
- numpy-base=1.19.2=py38hfa32c7d_0
- oauthlib=3.1.0=py_0
- olefile=0.46=py_0
- openssl=1.1.1k=h27cfd23_0 # (updated from 1.1.1j in May 26, 2021 maintenance update)
- packaging=20.4=py_0
- pandas=1.1.3=py38he6710b0_0
- paramiko=2.7.2=py_0
- parso=0.7.0=py_0
- patsy=0.5.1=py38_0
- pexpect=4.8.0=pyhd3eb1b0_3
- pickleshare=0.7.5=pyhd3eb1b0_1003
- pillow=8.0.1=py38he98fc37_0
- pip=20.2.4=py38h06a4308_0
- plotly=4.14.3=pyhd3eb1b0_0
- prompt-toolkit=3.0.8=py_0
- prompt_toolkit=3.0.8=0
- protobuf=3.13.0.1=py38he6710b0_1
- psutil=5.7.2=py38h7b6447c_0
- psycopg2=2.8.5=py38h3c74f83_1
- ptyprocess=0.6.0=pyhd3eb1b0_2
- pyasn1=0.4.8=py_0
- pyasn1-modules=0.2.8=py_0
- pycparser=2.20=py_2
- pygments=2.7.2=pyhd3eb1b0_0
- pyjwt=1.7.1=py38_0
- pynacl=1.4.0=py38h7b6447c_1
- pyodbc=4.0.30=py38he6710b0_0
- pyopenssl=19.1.0=pyhd3eb1b0_1
- pyparsing=2.4.7=pyhd3eb1b0_0
- pysocks=1.7.1=py38h06a4308_0
- python=3.8.8=hdb3f193_4 # (updated from 3.8.5 in May 26, 2021 maintenance update)
- python-dateutil=2.8.1=pyhd3eb1b0_0
- python-editor=1.0.4=py_0
- pytorch=1.7.1=py3.8_cpu_0
- pytz=2020.5=pyhd3eb1b0_0
- pyzmq=19.0.2=py38he6710b0_1
- readline=8.0=h7b6447c_0
- regex=2020.10.15=py38h7b6447c_0
- requests=2.24.0=py_0
- requests-oauthlib=1.3.0=py_0
- retrying=1.3.3=py_2
- rsa=4.7.2=pyhd3eb1b0_1
- s3transfer=0.3.4=pyhd3eb1b0_0
- scikit-learn=0.23.2=py38h0573a6f_0
- scipy=1.5.2=py38h0b6359f_0
- setuptools=50.3.1=py38h06a4308_1
- simplejson=3.17.2=py38h27cfd23_2
- six=1.15.0=py38h06a4308_0
- smmap=3.0.5=pyhd3eb1b0_0
- sqlite=3.33.0=h62c20be_0
- sqlparse=0.4.1=py_0
- statsmodels=0.12.0=py38h7b6447c_0
- tabulate=0.8.7=py38h06a4308_0
- threadpoolctl=2.1.0=pyh5ca1d4c_0
- tk=8.6.10=hbc83047_0
- torchvision=0.8.2=py38_cpu
- tornado=6.0.4=py38h7b6447c_1
- tqdm=4.50.2=py_0
- traitlets=5.0.5=pyhd3eb1b0_0
- typing-extensions=3.7.4.3=hd3eb1b0_0
- typing_extensions=3.7.4.3=pyh06a4308_0
- unixodbc=2.3.9=h7b6447c_0
- urllib3=1.25.11=py_0
- wcwidth=0.2.5=py_0
- websocket-client=0.57.0=py38_2
- werkzeug=1.0.1=pyhd3eb1b0_0
- wheel=0.35.1=pyhd3eb1b0_0
- wrapt=1.12.1=py38h7b6447c_1
- xz=5.2.5=h7b6447c_0
- yarl=1.6.3=py38h27cfd23_0
- zeromq=4.3.3=he6710b0_3
- zipp=3.4.0=pyhd3eb1b0_0
- zlib=1.2.11=h7b6447c_3
- zstd=1.4.5=h9ceee32_0
- pip:
- astunparse==1.6.3
- azure-core==1.11.0
- azure-storage-blob==12.7.1
- databricks-cli==0.14.1
- diskcache==5.2.1
- docker==4.4.4
- flatbuffers==1.12
- gast==0.3.3
- grpcio==1.32.0
- horovod==0.21.1
- joblibspark==0.3.0
- keras-preprocessing==1.1.2
- koalas==1.6.0
- llvmlite==0.35.0
- mleap==0.16.1
- mlflow==1.14.1
- msrest==0.6.21
- numba==0.52.0
- opt-einsum==3.3.0
- petastorm==0.9.8
- pyarrow==1.0.1
- pyyaml==5.4.1
- querystring-parser==1.2.4
- seaborn==0.10.0
- shap==0.38.1
- slicer==0.0.7
- spark-tensorflow-distributor==0.1.0
- tensorboard==2.4.1
- tensorboard-plugin-wit==1.8.0
- tensorflow-cpu==2.4.1
- tensorflow-estimator==2.4.0
- termcolor==1.1.0
- xgboost==1.3.3
prefix: /databricks/conda/envs/databricks-ml
Python-bibliotek i GPU-kluster
name: databricks-ml-gpu
channels:
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- absl-py=0.11.0=pyhd3eb1b0_1
- aiohttp=3.7.4=py38h27cfd23_1
- asn1crypto=1.4.0=py_0
- astor=0.8.1=py38h06a4308_0
- async-timeout=3.0.1=py38h06a4308_0
- attrs=20.3.0=pyhd3eb1b0_0
- backcall=0.2.0=pyhd3eb1b0_0
- bcrypt=3.2.0=py38h7b6447c_0
- blas=1.0=mkl
- blinker=1.4=py38h06a4308_0
- boto3=1.16.7=pyhd3eb1b0_0
- botocore=1.19.7=pyhd3eb1b0_0
- brotlipy=0.7.0=py38h27cfd23_1003
- c-ares=1.17.1=h27cfd23_0
- ca-certificates=2021.4.13=h06a4308_1 # (updated from 2021.1.19 in May 26, 2021 maintenance update)
- cachetools=4.2.1=pyhd3eb1b0_0
- certifi=2020.12.5=py38h06a4308_0
- cffi=1.14.3=py38h261ae71_2
- chardet=3.0.4=py38h06a4308_1003
- click=7.1.2=pyhd3eb1b0_0
- cloudpickle=1.6.0=py_0
- configparser=5.0.1=py_0
- cryptography=3.1.1=py38h1ba5d50_0
- cycler=0.10.0=py38_0
- cython=0.29.21=py38h2531618_0
- decorator=4.4.2=pyhd3eb1b0_0
- dill=0.3.2=py_0
- docutils=0.15.2=py38h06a4308_1
- entrypoints=0.3=py38_0
- flask=1.1.2=pyhd3eb1b0_0
- freetype=2.10.4=h5ab3b9f_0
- future=0.18.2=py38_1
- gitdb=4.0.5=py_0
- gitpython=3.1.12=pyhd3eb1b0_1
- google-auth=1.22.1=py_0
- google-auth-oauthlib=0.4.2=pyhd3eb1b0_2
- google-pasta=0.2.0=py_0
- grpcio=1.31.0=py38hf8bcb03_0
- gunicorn=20.0.4=py38_0
- h5py=2.10.0=py38h7918eee_0
- hdf5=1.10.4=hb1b8bf9_0
- icu=58.2=he6710b0_3
- idna=2.10=pyhd3eb1b0_0
- importlib-metadata=2.0.0=py_1
- intel-openmp=2019.4=243
- ipykernel=5.3.4=py38h5ca1d4c_0
- ipython=7.19.0=py38hb070fc8_1
- ipython_genutils=0.2.0=pyhd3eb1b0_1
- isodate=0.6.0=py_1
- itsdangerous=1.1.0=pyhd3eb1b0_0
- jedi=0.17.2=py38h06a4308_1
- jinja2=2.11.2=pyhd3eb1b0_0
- jmespath=0.10.0=py_0
- joblib=0.17.0=py_0
- jpeg=9b=h024ee3a_2
- jupyter_client=6.1.7=py_0
- jupyter_core=4.6.3=py38_0
- kiwisolver=1.3.0=py38h2531618_0
- krb5=1.17.1=h173b8e3_0
- lcms2=2.11=h396b838_0
- ld_impl_linux-64=2.33.1=h53a641e_7
- libedit=3.1.20191231=h14c3975_1
- libffi=3.3=he6710b0_2
- libgcc-ng=9.1.0=hdf63c60_0
- libgfortran-ng=7.3.0=hdf63c60_0
- libpng=1.6.37=hbc83047_0
- libpq=12.2=h20c2e04_0
- libprotobuf=3.13.0.1=hd408876_0
- libsodium=1.0.18=h7b6447c_0
- libstdcxx-ng=9.1.0=hdf63c60_0
- libtiff=4.1.0=h2733197_1
- lightgbm=3.1.1=py38h2531618_0
- lz4-c=1.9.2=heb0550a_3
- mako=1.1.3=py_0
- markdown=3.3.3=py38h06a4308_0
- markupsafe=1.1.1=py38h7b6447c_0
- matplotlib-base=3.2.2=py38hef1b27d_0
- mkl=2019.4=243
- mkl-service=2.3.0=py38he904b0f_0
- mkl_fft=1.2.0=py38h23d657b_0
- mkl_random=1.1.0=py38h962f231_0
- more-itertools=8.6.0=pyhd3eb1b0_0
- multidict=5.1.0=py38h27cfd23_2
- ncurses=6.2=he6710b0_1
- networkx=2.5=py_0
- nltk=3.5=py_0
- numpy=1.19.2=py38h54aff64_0
- numpy-base=1.19.2=py38hfa32c7d_0
- oauthlib=3.1.0=py_0
- olefile=0.46=py_0
- openssl=1.1.1k=h27cfd23_0 # (updated from 1.1.1i in May 26, 2021 maintenance update)
- packaging=20.4=py_0
- pandas=1.1.3=py38he6710b0_0
- paramiko=2.7.2=py_0
- parso=0.7.0=py_0
- patsy=0.5.1=py38_0
- pexpect=4.8.0=pyhd3eb1b0_3
- pickleshare=0.7.5=pyhd3eb1b0_1003
- pillow=8.0.1=py38he98fc37_0
- pip=20.2.4=py38h06a4308_0
- plotly=4.14.3=pyhd3eb1b0_0
- prompt-toolkit=3.0.8=py_0
- prompt_toolkit=3.0.8=0
- protobuf=3.13.0.1=py38he6710b0_1
- psutil=5.7.2=py38h7b6447c_0
- psycopg2=2.8.5=py38h3c74f83_1
- ptyprocess=0.6.0=pyhd3eb1b0_2
- pyasn1=0.4.8=py_0
- pyasn1-modules=0.2.8=py_0
- pycparser=2.20=py_2
- pygments=2.7.2=pyhd3eb1b0_0
- pyjwt=1.7.1=py38_0
- pynacl=1.4.0=py38h7b6447c_1
- pyodbc=4.0.30=py38he6710b0_0
- pyopenssl=19.1.0=pyhd3eb1b0_1
- pyparsing=2.4.7=pyhd3eb1b0_0
- pysocks=1.7.1=py38h06a4308_0
- python=3.8.8=hdb3f193_4 # (updated from 3.8.5 in May 26, 2021 maintenance update)
- python-dateutil=2.8.1=pyhd3eb1b0_0
- python-editor=1.0.4=py_0
- pytz=2020.5=pyhd3eb1b0_0
- pyzmq=19.0.2=py38he6710b0_1
- readline=8.0=h7b6447c_0
- regex=2020.10.15=py38h7b6447c_0
- requests=2.24.0=py_0
- requests-oauthlib=1.3.0=py_0
- retrying=1.3.3=py_2
- rsa=4.7.2=pyhd3eb1b0_1
- s3transfer=0.3.4=pyhd3eb1b0_0
- scikit-learn=0.23.2=py38h0573a6f_0
- scipy=1.5.2=py38h0b6359f_0
- setuptools=50.3.1=py38h06a4308_1
- simplejson=3.17.2=py38h27cfd23_2
- six=1.15.0=py38h06a4308_0
- smmap=3.0.5=pyhd3eb1b0_0
- sqlite=3.33.0=h62c20be_0
- sqlparse=0.4.1=py_0
- statsmodels=0.12.0=py38h7b6447c_0
- tabulate=0.8.7=py38h06a4308_0
- threadpoolctl=2.1.0=pyh5ca1d4c_0
- tk=8.6.10=hbc83047_0
- tornado=6.0.4=py38h7b6447c_1
- tqdm=4.50.2=py_0
- traitlets=5.0.5=pyhd3eb1b0_0
- typing-extensions=3.7.4.3=hd3eb1b0_0
- typing_extensions=3.7.4.3=pyh06a4308_0
- unixodbc=2.3.9=h7b6447c_0
- urllib3=1.25.11=py_0
- wcwidth=0.2.5=py_0
- websocket-client=0.57.0=py38_2
- werkzeug=1.0.1=pyhd3eb1b0_0
- wheel=0.35.1=pyhd3eb1b0_0
- wrapt=1.12.1=py38h7b6447c_1
- xz=5.2.5=h7b6447c_0
- yarl=1.6.3=py38h27cfd23_0
- zeromq=4.3.3=he6710b0_3
- zipp=3.4.0=pyhd3eb1b0_0
- zlib=1.2.11=h7b6447c_3
- zstd=1.4.5=h9ceee32_0
- pip:
- astunparse==1.6.3
- azure-core==1.11.0
- azure-storage-blob==12.7.1
- databricks-cli==0.14.1
- diskcache==5.2.1
- docker==4.4.4
- flatbuffers==1.12
- gast==0.3.3
- horovod==0.21.1
- joblibspark==0.3.0
- keras-preprocessing==1.1.2
- koalas==1.6.0
- llvmlite==0.35.0
- mleap==0.16.1
- mlflow==1.14.1
- msrest==0.6.21
- numba==0.52.0
- opt-einsum==3.3.0
- petastorm==0.9.8
- pyarrow==1.0.1
- pyyaml==5.4.1
- querystring-parser==1.2.4
- seaborn==0.10.0
- shap==0.38.1
- slicer==0.0.7
- spark-tensorflow-distributor==0.1.0
- tensorboard==2.4.1
- tensorboard-plugin-wit==1.8.0
- tensorflow==2.4.1
- tensorflow-estimator==2.4.0
- termcolor==1.1.0
- torch==1.7.1
- torchvision==0.8.2
- xgboost==1.3.3
prefix: /databricks/conda/envs/databricks-ml-gpu
Spark-paket som innehåller Python-moduler
Spark-paket | Python-modul | Version |
---|---|---|
graphframes | graphframes | 0.8.1-db2-spark3.1 |
R-bibliotek
R-biblioteken är identiska med R-biblioteken i Databricks Runtime 8.1.
Java- och Scala-bibliotek (Scala 2.12-kluster)
Förutom Java- och Scala-bibliotek i Databricks Runtime 8.1 innehåller Databricks Runtime 8.1 ML följande JAR:er:
CPU-kluster
Grupp-ID | Artefakt-ID | Version |
---|---|---|
com.typesafe.akka | akka-actor_2.12 | 2.5.23 |
ml.combust.mleap | mleap-databricks-runtime_2.12 | 0.17.3-4882dc3 |
ml.dmlc | xgboost4j-spark_2.12 | 1.3.1 |
ml.dmlc | xgboost4j_2.12 | 1.3.1 |
org.mlflow | mlflow-client | 1.14.1 |
org.scala-lang.modules | scala-java8-compat_2.12 | 0.8.0 |
org.tensorflow | spark-tensorflow-connector_2.12 | 1.15.0 |
GPU-kluster
Grupp-ID | Artefakt-ID | Version |
---|---|---|
com.typesafe.akka | akka-actor_2.12 | 2.5.23 |
ml.combust.mleap | mleap-databricks-runtime_2.12 | 0.17.3-4882dc3 |
ml.dmlc | xgboost4j-spark-gpu_2.12 | 1.3.1 |
ml.dmlc | xgboost4j-gpu_2.12 | 1.3.1 |
org.mlflow | mlflow-client | 1.14.1 |
org.scala-lang.modules | scala-java8-compat_2.12 | 0.8.0 |
org.tensorflow | spark-tensorflow-connector_2.12 | 1.15.0 |