Catatan
Akses ke halaman ini memerlukan otorisasi. Anda dapat mencoba masuk atau mengubah direktori.
Akses ke halaman ini memerlukan otorisasi. Anda dapat mencoba mengubah direktori.
Catatan
Dukungan untuk versi Databricks Runtime ini telah berakhir. Untuk tanggal akhir dukungan, lihat Riwayat akhir dukungan. Untuk semua versi Runtime Databricks yang didukung, lihat Versi dan kompatibilitas catatan rilis Databricks Runtime.
Databricks merilis versi ini pada Februari 2021.
Databricks Runtime 7.6 untuk Pembelajaran Mesin menyediakan lingkungan siap pakai untuk pembelajaran mesin dan ilmu data berdasarkan Databricks Runtime 7.6 (EoS). Runtime Databricks ML berisi banyak pustaka populer untuk pembelajaran mesin, termasuk TensorFlow, PyTorch, dan XGBoost. Ini juga mendukung pelatihan pembelajaran mendalam terdistribusi menggunakan Horovod.
Untuk informasi selengkapnya, termasuk instruksi untuk membuat kluster ML Runtime Databricks, lihat AI dan pembelajaran mesin di Databricks.
Untuk bantuan tentang migrasi dari Databricks Runtime 6.x, lihat Panduan migrasi Databricks Runtime 7.x (EoS).
Fitur baru dan perubahan besar
Databricks Runtime 7.6 ML dibangun di atas basis Databricks Runtime 7.6. Untuk informasi tentang apa yang baru di Databricks Runtime 7.6, termasuk Apache Spark MLlib dan SparkR, lihat catatan rilis Databricks Runtime 7.6 (EoS).
Pemendaman
- Tensorflow 1.x tidak akan didukung dalam rilis utama Databricks Runtime yang akan datang.
- Paket CUDA berikut tidak digunakan lagi dan akan dihapus dalam rilis utama Databricks Runtime yang akan datang:
- Alat Baris Perintah CUDA
- pengkompilasi CUDA
- cuda-cudart-dev
- cuda-manfft
- cuda-cufft-dev
- cuda-cuobjdump
- cuda-cupti
- cuda-curand
- cuda-curand-dev
- cuda-cusolver
- cuda-cusolver-dev
- cuda-cusparse
- cuda-cusparse-dev
- dokumentasi cuda
- cuda-driver-dev
- cuda-gdb
- penasihat-perpustakaan-cuda-gpu
- cuda-libraries-dev
- lisensi CUDA
- cuda-memcheck
- cuda-minimal-build
- cuda-misc-headers
- cuda-npp
- cuda-npp-dev
- cuda-nsight
- cuda-nvcc
- cuda-nvdisasm
- cuda-nvgraph
- cuda-nvgraph-dev
- cuda-nvjpeg
- cuda-nvjpeg-dev
- cuda-nvml-dev
- cuda-nvprune
- cuda-nvrtc-dev
- cuda-nvvp
- cuda-samples
- cuda-sanitizer-api
- CUDA Toolkit
- Alat CUDA
- Alat Visual CUDA
- freeglut3
- libcublas-dev
- libcudnn7-dev
- libdrm-dev
- libegl1
- libegm-mesa0
- libgbl1-mesa-dev
- libgbm1
- libgles1
- libgles2
- libglu1-mesa
- libglu1-mesa-dev
- libnccl-dev
- libnvinfer-dev
- libnvinfer-plugin-dev
- libopengl0
- libwayland-server0
- libx11-xcb-dev
- libxcb-dri2-0-dev
- libxcb-dri3-dev
- libxcb-glx0-dev
- libxcb-present-dev
- libxcb-randr0
- libxcb-randr0-dev
- libxcb-render0-dev
- libxcb-shape0-dev
- libxcb-sync-dev
- libxcb-xfixes0
- libxcb-xfixes0-dev
- libxdamage-dev
- libxext-dev
- libxfixes-dev
- libxi-dev
- libxmu-dev
- libxmu-headers
- libxshmfence-dev
- libxxf86vm-dev
- mesa-common-dev
- nsight-compute
- nsight-systems
- x11proto-damage-dev
- x11proto-fixes-dev
- x11proto-input-dev
- x11proto-xext-dev
- x11proto-xf86vidmode-dev
Perubahan besar pada lingkungan Python ML di Databricks Runtime
Lihat Databricks Runtime 7.6 (EoS) untuk perubahan besar pada lingkungan Databricks Runtime Python. Untuk daftar lengkap paket Python yang diinstal dan versinya, lihat Pustaka Python.
Paket Phyton ditingkatkan
- databricks-cli 0.14.0 -> 0.14.1
- koala 1.4.0 -> 1.5.0
- lightgbm 2.3.0 -> 3.1.1
- mlflow 1.12.1 -> 1.13.1
- plotly 4.12.0 -> 4.14.1
- pytorch 1.7.0 -> 1.7.1
- torchvision 0.8.1 -> 0.8.2
- xgboost 1.2.1 -> 1.3.1
Penyempurnaan
Integrasi XGBoost ke dalam PySpark (Pratinjau Umum)
Integrasi XGBoost dengan PySpark telah ditingkatkan. Paket sparkdl 2.1.0-db5
ini mencakup dua estimator pyspark ML baru, XgboostRegressor
dan XgboostClassifier
, yang memungkinkan pengguna untuk melatih model XGBoost di Alur PySpark ML.
Sebelum versi ini, XGBoost tidak terintegrasi dengan PySpark. Pengguna harus menggunakan xgboost4j-spark
di Scala atau memecahkan Pipeline PySpark ML, mengumpulkan Spark DataFrame pada driver sebagai DataFrame Pandas, dan menggunakan paket Python xgboost
. Lihat dokumentasi sparkdl API dan Menggunakan XGBoost di Azure Databricks untuk detail lebih lanjut.
Lingkungan sistem
Lingkungan sistem di Databricks Runtime 7.6 ML berbeda dari Databricks Runtime 7.6 berikut ini:
-
DBUtils: Databricks Runtime ML tidak berisi utilitas Pustaka (dbutils.library) (warisan).
Anda dapat menggunakan perintah
%pip
dan%conda
sebagai gantinya. Lihat Perpustakaan Python yang berlaku untuk notebook. - Untuk kluster GPU, Databricks Runtime ML menyertakan pustaka GPU NVIDIA berikut:
- CUDA 10.1 Pembaruan 2
- cuDNN 7.6.5
- NCCL 2.7.3
- TensorRT 6.0.1
Pustaka
Bagian berikut mencantumkan pustaka yang disertakan dalam Databricks Runtime 7.6 ML yang berbeda dari yang termasuk dalam Databricks Runtime 7.6.
Di bagian ini:
- Perpustakaan terkemuka
- Perpustakaan Python
- Pustaka bahasa pemrograman R
- Pustaka Java dan Scala (Kluster Scala 2.12)
Pustaka tingkat atas
Databricks Runtime 7.6 ML mencakup pustaka tingkat atas berikut:
- GraphFrame
- Horovod dan HorovodRunner
- MLflow
- PyTorch
- spark-tensorflow-connector
- TensorFlow
- TensorBoard
Perpustakaan Python
Databricks Runtime 7.6 ML menggunakan Conda untuk manajemen paket Python dan mencakup banyak paket ML populer.
Selain paket yang ditentukan di lingkungan Conda di bagian berikut, Databricks Runtime 7.6 ML juga menginstal paket berikut:
- hyperopt 0.2.5.db1
- sparkdl 2.1.0-db5
Pustaka Python di kluster CPU
name: databricks-ml
channels:
- pytorch
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- absl-py=0.9.0=py37_0
- asn1crypto=1.3.0=py37_1
- astor=0.8.0=py37_0
- backcall=0.1.0=py37_0
- backports=1.0=pyhd3eb1b0_2
- bcrypt=3.2.0=py37h7b6447c_0
- blas=1.0=mkl
- blinker=1.4=py37_0
- boto3=1.12.0=py_0
- botocore=1.15.0=py_0
- c-ares=1.17.1=h27cfd23_0
- ca-certificates=2021.1.19=h06a4308_1 # (updated from h06a4308_0 in May 26, 2021 maintenance update)
- cachetools=4.2.0=pyhd3eb1b0_0
- certifi=2020.12.5=py37h06a4308_0
- cffi=1.14.0=py37he30daa8_1 # (updated from py37h2e261b9_0 in May 26, 2021 maintenance update)
- chardet=3.0.4=py37h06a4308_1003
- click=7.0=py37_0
- cloudpickle=1.4.1=py_0
- configparser=3.7.4=py37_0
- cpuonly=1.0=0
- cryptography=2.8=py37h1ba5d50_0
- cycler=0.10.0=py37_0
- cython=0.29.15=py37he6710b0_0
- decorator=4.4.1=py_0
- dill=0.3.1.1=py37_1
- docutils=0.15.2=py37_0
- entrypoints=0.3=py37_0
- flask=1.1.1=py_1
- freetype=2.9.1=h8a8886c_1
- future=0.18.2=py37_1
- gast=0.3.3=py_0
- gitdb=4.0.5=py_0
- gitpython=3.1.0=py_0
- google-auth=1.11.2=py_0
- google-auth-oauthlib=0.4.1=py_2
- google-pasta=0.2.0=py_0
- grpcio=1.27.2=py37hf8bcb03_0
- gunicorn=20.0.4=py37_0
- h5py=2.10.0=py37h7918eee_0
- hdf5=1.10.4=hb1b8bf9_0
- icu=58.2=he6710b0_3
- idna=2.8=py37_0
- intel-openmp=2020.0=166
- ipykernel=5.1.4=py37h39e3cac_0
- ipython=7.12.0=py37h5ca1d4c_0
- ipython_genutils=0.2.0=pyhd3eb1b0_1
- isodate=0.6.0=py_1
- itsdangerous=1.1.0=py37_0
- jedi=0.17.2=py37h06a4308_1
- jinja2=2.11.1=py_0
- jmespath=0.10.0=py_0
- joblib=0.14.1=py_0
- jpeg=9b=h024ee3a_2
- jupyter_client=5.3.4=py37_0
- jupyter_core=4.6.1=py37_0
- kiwisolver=1.1.0=py37he6710b0_0
- krb5=1.17.1=h173b8e3_0 # (updated from 1.16.4 in May 26, 2021 maintenance update)
- ld_impl_linux-64=2.33.1=h53a641e_7
- libedit=3.1.20181209=hc058e9b_0
- libffi=3.3=he6710b0_2 # (updated from 3.2.1 in May 26, 2021 maintenance update)
- libgcc-ng=9.1.0=hdf63c60_0
- libgfortran-ng=7.3.0=hdf63c60_0
- libpng=1.6.37=hbc83047_0
- libpq=12.2=h20c2e04_0 # (updated from 11.2 in May 26, 2021 maintenance update)
- libprotobuf=3.11.4=hd408876_0
- libsodium=1.0.16=h1bed415_0
- libstdcxx-ng=9.1.0=hdf63c60_0
- libtiff=4.1.0=h2733197_0
- libuv=1.40.0=h7b6447c_0
- lightgbm=3.1.1=py37h2531618_0
- lz4-c=1.8.1.2=h14c3975_0
- mako=1.1.2=py_0
- markdown=3.1.1=py37_0
- markupsafe=1.1.1=py37h14c3975_1
- matplotlib-base=3.1.3=py37hef1b27d_0
- mkl=2020.0=166
- mkl-service=2.3.0=py37he8ac12f_0
- mkl_fft=1.0.15=py37ha843d7b_0
- mkl_random=1.1.0=py37hd6b4f25_0
- ncurses=6.2=he6710b0_1
- networkx=2.4=py_1
- ninja=1.10.2=py37hff7bd54_0
- nltk=3.4.5=py37_0
- numpy=1.18.1=py37h4f9e942_0
- numpy-base=1.18.1=py37hde5b4d6_1
- oauthlib=3.1.0=py_0
- olefile=0.46=py37_0
- openssl=1.1.1k=h27cfd23_0 # (updated from 1.1.1i in May 26, 2021 maintenance update)
- packaging=20.1=py_0
- pandas=1.0.1=py37h0573a6f_0
- paramiko=2.7.1=py_0
- parso=0.7.0=py_0
- patsy=0.5.1=py37_0
- pexpect=4.8.0=pyhd3eb1b0_3
- pickleshare=0.7.5=pyhd3eb1b0_1003
- pillow=7.0.0=py37hb39fc2d_0
- pip=20.0.2=py37_3
- plotly=4.14.1=pyhd3eb1b0_0
- prompt_toolkit=3.0.3=py_0
- protobuf=3.11.4=py37he6710b0_0
- psutil=5.6.7=py37h7b6447c_0
- psycopg2=2.8.6=py37h3c74f83_1 # (updated from 2.8.4 in May 26, 2021 maintenance update)
- ptyprocess=0.6.0=pyhd3eb1b0_2
- pyasn1=0.4.8=py_0
- pyasn1-modules=0.2.8=py_0
- pycparser=2.19=py37_0
- pygments=2.5.2=py_0
- pyjwt=2.0.1=py37h06a4308_0
- pynacl=1.3.0=py37h7b6447c_0
- pyodbc=4.0.30=py37he6710b0_0
- pyopenssl=19.1.0=pyhd3eb1b0_1
- pyparsing=2.4.6=py_0
- pysocks=1.7.1=py37_1
- python=3.7.10=hdb3f193_0 # (updated from 3.7.6 in May 26, 2021 maintenance update)
- python-dateutil=2.8.1=py_0
- python-editor=1.0.4=py_0
- pytorch=1.7.1=py3.7_cpu_0
- pytz=2019.3=py_0
- pyzmq=18.1.1=py37he6710b0_0
- readline=8.1=h27cfd23_0 # (updated from 7.0 in May 26, 2021 maintenance update)
- requests=2.22.0=py37_1
- requests-oauthlib=1.3.0=py_0
- retrying=1.3.3=py37_2
- rsa=4.0=py_0
- s3transfer=0.3.4=pyhd3eb1b0_0
- scikit-learn=0.22.1=py37hd81dba3_0
- scipy=1.4.1=py37h0b6359f_0
- setuptools=45.2.0=py37_0
- simplejson=3.17.0=py37h7b6447c_0
- six=1.14.0=py37h06a4308_0
- smmap=3.0.4=py_0
- sqlite=3.35.4=hdfb4753_0 # (updated from 3.31.1 in May 26, 2021 maintenance update)
- sqlparse=0.4.1=py_0
- statsmodels=0.11.0=py37h7b6447c_0
- tabulate=0.8.3=py37_0
- tk=8.6.10=hbc83047_0 # (updated from 8.6.8 in May 26, 2021 maintenance update)
- torchvision=0.8.2=py37_cpu
- tornado=6.0.3=py37h7b6447c_3
- tqdm=4.42.1=py_0
- traitlets=4.3.3=py37_0
- typing_extensions=3.7.4.3=py_0
- unixodbc=2.3.7=h14c3975_0
- urllib3=1.25.8=py37_0
- wcwidth=0.1.8=py_0
- websocket-client=0.56.0=py37_0
- werkzeug=1.0.0=py_0
- wheel=0.34.2=py37_0
- wrapt=1.11.2=py37h7b6447c_0
- xz=5.2.5=h7b6447c_0 # (updated from 5.2.4 in May 26, 2021 maintenance update)
- zeromq=4.3.1=he6710b0_3
- zlib=1.2.11=h7b6447c_3
- zstd=1.3.7=h0b5b093_0
- pip:
- astunparse==1.6.3
- azure-core==1.10.0
- azure-storage-blob==12.7.0
- databricks-cli==0.14.1
- diskcache==5.1.0
- docker==4.4.1
- gorilla==0.3.0
- horovod==0.20.3
- joblibspark==0.3.0
- keras-preprocessing==1.1.2
- koalas==1.5.0
- mleap==0.16.1
- mlflow==1.13.1
- msrest==0.6.19
- opt-einsum==3.3.0
- petastorm==0.9.7
- pyarrow==1.0.1
- pyyaml==5.4
- querystring-parser==1.2.4
- seaborn==0.10.0
- spark-tensorflow-distributor==0.1.0
- tensorboard==2.3.0
- tensorboard-plugin-wit==1.8.0
- tensorflow-cpu==2.3.1
- tensorflow-estimator==2.3.0
- termcolor==1.1.0
- xgboost==1.3.1
prefix: /databricks/conda/envs/databricks-ml
Perpustakaan Python di kluster GPU
name: databricks-ml-gpu
channels:
- pytorch
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- absl-py=0.9.0=py37_0
- asn1crypto=1.3.0=py37_1
- astor=0.8.0=py37_0
- backcall=0.1.0=py37_0
- backports=1.0=pyhd3eb1b0_2
- bcrypt=3.2.0=py37h7b6447c_0
- blas=1.0=mkl
- blinker=1.4=py37_0
- boto3=1.12.0=py_0
- botocore=1.15.0=py_0
- c-ares=1.17.1=h27cfd23_0
- ca-certificates=2021.1.19=h06a4308_1 # (updated from h06a4308_0 in May 26, 2021 maintenance update)
- cachetools=4.2.0=pyhd3eb1b0_0
- certifi=2020.12.5=py37h06a4308_0
- cffi=1.14.0=py37he30daa8_1 # (updated from py37h2e261b9_0 in May 26, 2021 maintenance update)
- chardet=3.0.4=py37h06a4308_1003
- click=7.0=py37_0
- cloudpickle=1.4.1=py_0
- configparser=3.7.4=py37_0
- cryptography=2.8=py37h1ba5d50_0
- cudatoolkit=10.1.243=h6bb024c_0
- cycler=0.10.0=py37_0
- cython=0.29.15=py37he6710b0_0
- decorator=4.4.1=py_0
- dill=0.3.1.1=py37_1
- docutils=0.15.2=py37_0
- entrypoints=0.3=py37_0
- flask=1.1.1=py_1
- freetype=2.9.1=h8a8886c_1
- future=0.18.2=py37_1
- gast=0.3.3=py_0
- gitdb=4.0.5=py_0
- gitpython=3.1.0=py_0
- google-auth=1.11.2=py_0
- google-auth-oauthlib=0.4.1=py_2
- google-pasta=0.2.0=py_0
- grpcio=1.27.2=py37hf8bcb03_0
- gunicorn=20.0.4=py37_0
- h5py=2.10.0=py37h7918eee_0
- hdf5=1.10.4=hb1b8bf9_0
- icu=58.2=he6710b0_3
- idna=2.8=py37_0
- intel-openmp=2020.0=166
- ipykernel=5.1.4=py37h39e3cac_0
- ipython=7.12.0=py37h5ca1d4c_0
- ipython_genutils=0.2.0=pyhd3eb1b0_1
- isodate=0.6.0=py_1
- itsdangerous=1.1.0=py37_0
- jedi=0.17.2=py37h06a4308_1
- jinja2=2.11.1=py_0
- jmespath=0.10.0=py_0
- joblib=0.14.1=py_0
- jpeg=9b=h024ee3a_2
- jupyter_client=5.3.4=py37_0
- jupyter_core=4.6.1=py37_0
- kiwisolver=1.1.0=py37he6710b0_0
- krb5=1.17.1=h173b8e3_0 # (updated from 1.16.4 in May 26, 2021 maintenance update)
- ld_impl_linux-64=2.33.1=h53a641e_7
- libedit=3.1.20181209=hc058e9b_0
- libffi=3.3=he6710b0_2 # (updated from 3.2.1 in May 26, 2021 maintenance update)
- libgcc-ng=9.1.0=hdf63c60_0
- libgfortran-ng=7.3.0=hdf63c60_0
- libpng=1.6.37=hbc83047_0
- libpq=12.2=h20c2e04_0 # (updated from 11.2 in May 26, 2021 maintenance update)
- libprotobuf=3.11.4=hd408876_0
- libsodium=1.0.16=h1bed415_0
- libstdcxx-ng=9.1.0=hdf63c60_0
- libtiff=4.1.0=h2733197_0
- libuv=1.40.0=h7b6447c_0
- lightgbm=3.1.1=py37h2531618_0
- lz4-c=1.8.1.2=h14c3975_0
- mako=1.1.2=py_0
- markdown=3.1.1=py37_0
- markupsafe=1.1.1=py37h14c3975_1
- matplotlib-base=3.1.3=py37hef1b27d_0
- mkl=2020.0=166
- mkl-service=2.3.0=py37he8ac12f_0
- mkl_fft=1.0.15=py37ha843d7b_0
- mkl_random=1.1.0=py37hd6b4f25_0
- ncurses=6.2=he6710b0_1
- networkx=2.4=py_1
- ninja=1.10.2=py37hff7bd54_0
- nltk=3.4.5=py37_0
- numpy=1.18.1=py37h4f9e942_0
- numpy-base=1.18.1=py37hde5b4d6_1
- oauthlib=3.1.0=py_0
- olefile=0.46=py37_0
- openssl=1.1.1k=h27cfd23_0 # (updated from 1.1.1i in May 26, 2021 maintenance update)
- packaging=20.1=py_0
- pandas=1.0.1=py37h0573a6f_0
- paramiko=2.7.1=py_0
- parso=0.7.0=py_0
- patsy=0.5.1=py37_0
- pexpect=4.8.0=pyhd3eb1b0_3
- pickleshare=0.7.5=pyhd3eb1b0_1003
- pillow=7.0.0=py37hb39fc2d_0
- pip=20.0.2=py37_3
- plotly=4.14.1=pyhd3eb1b0_0
- prompt_toolkit=3.0.3=py_0
- protobuf=3.11.4=py37he6710b0_0
- psutil=5.6.7=py37h7b6447c_0
- psycopg2=2.8.6=py37h3c74f83_1 # (updated from 2.8.4 in May 26, 2021 maintenance update)
- ptyprocess=0.6.0=pyhd3eb1b0_2
- pyasn1=0.4.8=py_0
- pyasn1-modules=0.2.8=py_0
- pycparser=2.19=py37_0
- pygments=2.5.2=py_0
- pyjwt=2.0.1=py37h06a4308_0
- pynacl=1.3.0=py37h7b6447c_0
- pyodbc=4.0.30=py37he6710b0_0
- pyopenssl=19.1.0=pyhd3eb1b0_1
- pyparsing=2.4.6=py_0
- pysocks=1.7.1=py37_1
- python=3.7.10=hdb3f193_0 # (updated from 3.7.6 in May 26, 2021 maintenance update)
- python-dateutil=2.8.1=py_0
- python-editor=1.0.4=py_0
- pytorch=1.7.1=py3.7_cuda10.1.243_cudnn7.6.3_0
- pytz=2019.3=py_0
- pyzmq=18.1.1=py37he6710b0_0
- readline=8.1=h27cfd23_0 # (updated from 7.0 in May 26, 2021 maintenance update)
- requests=2.22.0=py37_1
- requests-oauthlib=1.3.0=py_0
- retrying=1.3.3=py37_2
- rsa=4.0=py_0
- s3transfer=0.3.4=pyhd3eb1b0_0
- scikit-learn=0.22.1=py37hd81dba3_0
- scipy=1.4.1=py37h0b6359f_0
- setuptools=45.2.0=py37_0
- simplejson=3.17.0=py37h7b6447c_0
- six=1.14.0=py37h06a4308_0
- smmap=3.0.4=py_0
- sqlite=3.35.4=hdfb4753_0 # (updated from 3.31.1 in May 26, 2021 maintenance update)
- sqlparse=0.4.1=py_0
- statsmodels=0.11.0=py37h7b6447c_0
- tabulate=0.8.3=py37_0
- tk=8.6.10=hbc83047_0 # (updated from 8.6.8 in May 26, 2021 maintenance update)
- torchvision=0.8.2=py37_cu101
- tornado=6.0.3=py37h7b6447c_3
- tqdm=4.42.1=py_0
- traitlets=4.3.3=py37_0
- typing_extensions=3.7.4.3=py_0
- unixodbc=2.3.7=h14c3975_0
- urllib3=1.25.8=py37_0
- wcwidth=0.1.8=py_0
- websocket-client=0.56.0=py37_0
- werkzeug=1.0.0=py_0
- wheel=0.34.2=py37_0
- wrapt=1.11.2=py37h7b6447c_0
- xz=5.2.5=h7b6447c_0 # (updated from 5.2.4 in May 26, 2021 maintenance update)
- zeromq=4.3.1=he6710b0_3
- zlib=1.2.11=h7b6447c_3
- zstd=1.3.7=h0b5b093_0
- pip:
- astunparse==1.6.3
- azure-core==1.10.0
- azure-storage-blob==12.7.0
- databricks-cli==0.14.1
- diskcache==5.1.0
- docker==4.4.1
- gorilla==0.3.0
- horovod==0.20.3
- joblibspark==0.3.0
- keras-preprocessing==1.1.2
- koalas==1.5.0
- mleap==0.16.1
- mlflow==1.13.1
- msrest==0.6.19
- opt-einsum==3.3.0
- petastorm==0.9.7
- pyarrow==1.0.1
- pyyaml==5.4
- querystring-parser==1.2.4
- seaborn==0.10.0
- spark-tensorflow-distributor==0.1.0
- tensorboard==2.3.0
- tensorboard-plugin-wit==1.8.0
- tensorflow==2.3.1
- tensorflow-estimator==2.3.0
- termcolor==1.1.0
- xgboost==1.3.1
prefix: /databricks/conda/envs/databricks-ml-gpu
Paket Spark yang berisi modul Python
Paket Spark | Modul Python | Versi |
---|---|---|
graphframe | graphframe | 0.8.1-db1-spark3.0 |
Pustaka R
Pustaka R identik dengan Pustaka R di Databricks Runtime 7.6.
Pustaka Java dan Scala (Kluster Scala 2.12)
Selain pustaka Java dan Scala di Databricks Runtime 7.6, Databricks Runtime 7.6 ML berisi JAR berikut ini:
Kluster CPU
ID Grup | ID Artefak | Versi |
---|---|---|
com.typeafe.akka | akka-actor_2.12 | 2.5.23 |
ml.combust.mleap | mleap-databricks-runtime_2.12 | 0.17.3-4882dc3 |
ml.dmlc | xgboost4j-spark_2.12 | 1.2.0 |
ml.dmlc | xgboost4j_2.12 | 1.2.0 |
org.mlflow | mlflow-client | 1.13.1 |
org.scala-lang.modules | scala-java8-compat_2.12 | 0.8.0 |
org.tensorflow | spark-tensorflow-connector_2.12 | 1.15.0 |
Kluster GPU
ID Grup | ID Artefak | Versi |
---|---|---|
com.typeafe.akka | akka-actor_2.12 | 2.5.23 |
ml.combust.mleap | mleap-databricks-runtime_2.12 | 0.17.3-4882dc3 |
ml.dmlc | xgboost4j-spark-gpu_2.12 | 1.2.0 |
ml.dmlc | xgboost4j-gpu_2.12 | 1.2.0 |
org.mlflow | mlflow-client | 1.13.1 |
org.scala-lang.modules | scala-java8-compat_2.12 | 0.8.0 |
org.tensorflow | spark-tensorflow-connector_2.12 | 1.15.0 |