Databricks Runtime 5.5 LTS pro ML (nepodporované)
Databricks vydala tuto image v červenci 2019. Podpora skončila 27. července 2021. Rozšířená podpora Databricks Runtime 5.5 ML (nepodporovaná) rozšiřuje podporu 5.5 ML do prosince 2021. Místo zastaralé distribuce Ubuntu 16.04.6 LTS použité v původní verzi Databricks Runtime 5.5 ML LTS používá Ubuntu 16.04.6 LTS. Podpora Ubuntu 16.04.6 LTS skončila 1. dubna 2021.
Databricks Runtime 5.5 LTS pro machine Učení poskytuje připravené prostředí pro strojové učení a datové vědy založené na databricks Runtime 5.5 LTS (nepodporované). Databricks Runtime ML obsahuje mnoho oblíbených knihoven strojového učení, včetně TensorFlow, PyTorch, Keras a XGBoost. Podporuje také distribuované trénování hlubokého učení pomocí Horovodu.
Další informace, včetně pokynů k vytvoření clusteru Databricks Runtime ML, najdete v tématu AI a machine Učení v Databricks.
Nové funkce
Databricks Runtime 5.5 LTS pro machine Učení je postaven na databricks Runtime 5.5 LTS. Informace o novinkách v Databricks Runtime 5.5 LTS najdete v poznámkách k verzi Databricks Runtime 5.5 LTS (nepodporované ).
Kromě aktualizací knihoven přináší Databricks Runtime 5.5 LTS pro machine Učení následující nové funkce:
- Přidání balíčku Pythonu MLflow 1.0
Vylepšení
Upgradované knihovny strojového učení
- TensorFlow upgradoval z verze 1.12.0 na verzi 1.13.1
- PyTorch upgradoval z verze 0.4.1 na verzi 1.1.0
- scikit-learn upgradován z verze 0.19.1 na 0.20.3
Operace s jedním uzlem pro HorovodRunner
Povolili jsme HorovodRunner, aby běžel jenom na uzlu ovladače. Pokud chcete dříve použít HorovodRunner, museli byste spustit ovladač a alespoň jeden pracovní uzel. Díky této změně teď můžete trénování distribuovat v jednom uzlu (tj. v uzlu s více GPU), a tudíž efektivněji používat výpočetní prostředky.
Vyřazení z provozu
V knihovně hyperopt jsme zastaralí následující vlastnosti hyperopt.SparkTrials
:
SparkTrials.successful_trials_count
SparkTrials.failed_trials_count
SparkTrials.cancelled_trials_count
SparkTrials.total_trials_count
a nahradil vlastnosti následujícími funkcemi:
SparkTrials.count_successful_trials()
SparkTrials.count_failed_trials()
SparkTrials.count_cancelled_trials()
SparkTrials.count_total_trials()
Prostředí systému
Systémové prostředí v Databricks Runtime 5.5 LTS pro machine Učení se liší od Databricks Runtime 5.5 následujícím způsobem:
- Python: 3.6.5 pro clustery Python 3 a 2.7.15 pro clustery Python 2.
- DBUtils: Neobsahuje nástroj Knihovny (dbutils.library) (starší verze).
- Pro clustery GPU následující knihovny NVIDIA GPU:
- CUDA 10.0
- CUDNN 7.6.0
Knihovny
Následující části obsahují seznam knihoven zahrnutých v Databricks Runtime 5.5 LTS pro machine Učení, které se liší od knihoven zahrnutých v Databricks Runtime 5.5.
Knihovny nejvyšší úrovně
Databricks Runtime 5.5 LTS pro machine Učení obsahuje následující knihovny nejvyšší úrovně:
Knihovny Pythonu
Databricks Runtime 5.5 LTS pro machine Učení používá Ke správě balíčků Pythonu Conda. V důsledku toho jsou v nainstalovaných knihovnách Pythonu v porovnání s Modulem Databricks Runtime velké rozdíly. Následující části popisují prostředí Conda pro prostředí Databricks Runtime 5.5 LTS pro clustery machine Učení s využitím Pythonu 2 nebo 3 a procesoru nebo počítačů s podporou GPU.
Python 3 v clusterech procesoru
name: null
channels:
- pytorch
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- _py-xgboost-mutex=2.0=cpu_0
- _tflow_select=2.3.0=mkl
- absl-py=0.7.1=py36_0
- asn1crypto=0.24.0=py36_0
- astor=0.7.1=py36_0
- backcall=0.1.0=py36_0
- backports=1.0=py_2
- bcrypt=3.1.6=py36h7b6447c_0
- blas=1.0=mkl
- bleach=2.1.3=py36_0
- boto=2.48.0=py36_1
- boto3=1.7.62=py36h28b3542_1
- botocore=1.10.62=py36h28b3542_0
- ca-certificates=2018.03.07=0
- certifi=2018.4.16=py36_0
- cffi=1.11.5=py36he75722e_1
- chardet=3.0.4=py36_1
- click=7.0=py36_0
- cloudpickle=0.8.0=py36_0
- colorama=0.3.9=py36h489cec4_0
- configparser=3.7.3=py36_1
- cryptography=2.2.2=py36h14c3975_0
- cycler=0.10.0=py36h93f1223_0
- cython=0.28.2=py36h14c3975_0
- decorator=4.3.0=py36_0
- docutils=0.14=py36hb0f60f5_0
- entrypoints=0.2.3=py36_2
- et_xmlfile=1.0.1=py36hd6bccc3_0
- flask=1.0.2=py36_1
- freetype=2.8=hab7d2ae_1
- gast=0.2.2=py36_0
- gitdb2=2.0.5=py36_0
- gitpython=2.1.11=py36_0
- gmp=6.1.2=h6c8ec71_1
- grpcio=1.12.1=py36hdbcaa40_0
- gunicorn=19.9.0=py36_0
- h5py=2.8.0=py36h989c5e5_3
- hdf5=1.10.2=hba1933b_1
- html5lib=1.0.1=py36_0
- icu=58.2=h9c2bf20_1
- idna=2.6=py36h82fb2a8_1
- intel-openmp=2018.0.0=8
- ipython=6.4.0=py36_1
- ipython_genutils=0.2.0=py36_0
- itsdangerous=0.24=py36_1
- jdcal=1.4=py36_0
- jedi=0.12.0=py36_1
- jinja2=2.10=py36_0
- jmespath=0.9.4=py_0
- jpeg=9b=h024ee3a_2
- jsonschema=2.6.0=py36_0
- jupyter_client=5.2.3=py36_0
- jupyter_core=4.4.0=py36_0
- keras=2.2.4=0
- keras-applications=1.0.8=py_0
- keras-base=2.2.4=py36_0
- keras-preprocessing=1.1.0=py_1
- krb5=1.16.1=hc83ff2d_6
- libedit=3.1.20170329=h6b74fdf_2
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.3.0=hdf63c60_0
- libgfortran-ng=7.2.0=hdf63c60_3
- libpng=1.6.34=hb9fc6fc_0
- libpq=10.4=h1ad7b7a_0
- libprotobuf=3.8.0=hd408876_0
- libsodium=1.0.16=h1bed415_0
- libstdcxx-ng=7.3.0=hdf63c60_0
- libtiff=4.0.9=he85c1e1_2
- libxgboost=0.90=he6710b0_0
- libxml2=2.9.8=h26e45fe_1
- libxslt=1.1.32=h1312cb7_0
- llvmlite=0.23.1=py36hdbcaa40_0
- lxml=4.2.1=py36h23eabaa_0
- mako=1.0.10=py_0
- markdown=3.1.1=py36_0
- markupsafe=1.0=py36h14c3975_1
- mistune=0.8.3=py36h14c3975_1
- mkl=2019.4=243
- mkl_fft=1.0.12=py36ha843d7b_0
- mkl_random=1.0.2=py36hd81dba3_0
- mock=3.0.5=py36_0
- msgpack-python=0.5.6=py36h6bb024c_1
- nbconvert=5.3.1=py36_0
- nbformat=4.4.0=py36h31c9010_0
- ncurses=6.1=he6710b0_1
- ninja=1.9.0=py36hfd86e86_0
- numba=0.38.0=py36h637b7d7_0
- numpy=1.16.2=py36h7e9f1db_0
- numpy-base=1.16.2=py36hde5b4d6_0
- olefile=0.45.1=py36_0
- openpyxl=2.5.3=py36_0
- openssl=1.0.2o=h14c3975_1
- pandas=0.23.0=py36h637b7d7_0
- pandocfilters=1.4.2=py36_1
- paramiko=2.4.2=py36_0
- parso=0.2.0=py36_0
- pathlib2=2.3.2=py36_0
- patsy=0.5.0=py36_0
- pexpect=4.5.0=py36_0
- pickleshare=0.7.4=py36_0
- pillow=5.1.0=py36h3deb7b8_0
- pip=10.0.1=py36_0
- ply=3.11=py36_0
- prompt_toolkit=1.0.15=py36h17d85b1_0
- protobuf=3.8.0=py36he6710b0_0
- psycopg2=2.7.5=py36hb7f436b_0
- ptyprocess=0.5.2=py36h69acd42_0
- py-xgboost=0.90=py36he6710b0_0
- py-xgboost-cpu=0.90=py36_0
- pyasn1=0.4.5=py_0
- pycparser=2.18=py36_1
- pygments=2.2.0=py36_0
- pynacl=1.3.0=py36h7b6447c_0
- pyopenssl=18.0.0=py36_0
- pyparsing=2.2.0=py36_1
- pysocks=1.6.8=py36_0
- python=3.6.5=hc3d631a_2
- python-dateutil=2.7.3=py36_0
- python-editor=1.0.4=py_0
- pytz=2018.4=py36_0
- pyyaml=5.1=py36h7b6447c_0
- pyzmq=17.0.0=py36h14c3975_3
- readline=7.0=h7b6447c_5
- requests=2.18.4=py36he2e5f8d_1
- s3transfer=0.1.13=py36_0
- scikit-learn=0.20.3=py36hd81dba3_0
- scipy=1.1.0=py36h7c811a0_2
- setuptools=39.1.0=py36_0
- simplegeneric=0.8.1=py36_2
- simplejson=3.16.0=py36h14c3975_0
- singledispatch=3.4.0.3=py36_0
- six=1.11.0=py36_1
- smmap2=2.0.5=py36_0
- sqlite=3.23.1=he433501_0
- sqlparse=0.3.0=py_0
- statsmodels=0.9.0=py36h035aef0_0
- tabulate=0.8.3=py36_0
- tensorboard=1.13.1=py36hf484d3e_0
- tensorflow=1.13.1=mkl_py36h27d456a_0
- tensorflow-base=1.13.1=mkl_py36h7ce6ba3_0
- tensorflow-estimator=1.13.0=py_0
- tensorflow-mkl=1.13.1=h4fcabd2_0
- termcolor=1.1.0=py36_1
- testpath=0.3.1=py36h8cadb63_0
- tk=8.6.7=hc745277_3
- tornado=5.0.2=py36h14c3975_0
- traitlets=4.3.2=py36_0
- urllib3=1.22=py36hbe7ace6_0
- virtualenv=16.0.0=py36_0
- wcwidth=0.1.7=py36hdf4376a_0
- webencodings=0.5.1=py36_1
- werkzeug=0.14.1=py36_0
- wheel=0.31.1=py36_0
- wrapt=1.11.1=py36h7b6447c_0
- xz=5.2.4=h14c3975_4
- yaml=0.1.7=had09818_2
- zeromq=4.2.5=hf484d3e_1
- zlib=1.2.11=h7b6447c_3
- pytorch-cpu=1.1.0=py3.6_cpu_0
- torchvision-cpu=0.3.0=py36_cuNone_1
- pip:
- databricks-cli==0.8.7
- docker==4.0.2
- fusepy==2.0.4
- future==0.17.1
- horovod==0.16.4
- hyperopt==0.1.2.db6
- kiwisolver==1.1.0
- matplotlib==2.2.2
- mleap==0.8.1
- mlflow==1.0.0
- msgpack==0.5.6
- networkx==2.2
- nose==1.3.7
- nose-exclude==0.5.0
- psutil==5.6.3
- pyarrow==0.13.0
- pymongo==3.8.0
- querystring-parser==1.2.3
- seaborn==0.8.1
- tensorboardx==1.7
- torchvision==0.3.0
- tqdm==4.32.2
- websocket-client==0.56.0
prefix: /databricks/python3
Python 3 v clusterech GPU
name: null
channels:
- pytorch
- Databricks
- defaults
dependencies:
- tensorflow=1.13.1.db1=gpu_py36h2903d8e_0
- tensorflow-base=1.13.1.db1=gpu_py36he292aa2_0
- tensorflow-gpu=1.13.1.db1=h0d30ee6_0
- _libgcc_mutex=0.1=main
- _py-xgboost-mutex=1.0=gpu_0
- _tflow_select=2.1.0=gpu
- absl-py=0.7.1=py36_0
- asn1crypto=0.24.0=py36_0
- astor=0.7.1=py36_0
- backcall=0.1.0=py36_0
- backports=1.0=py_2
- bcrypt=3.1.6=py36h7b6447c_0
- blas=1.0=mkl
- bleach=2.1.3=py36_0
- boto=2.48.0=py36_1
- boto3=1.7.62=py36h28b3542_1
- botocore=1.10.62=py36h28b3542_0
- ca-certificates=2018.03.07=0
- certifi=2018.4.16=py36_0
- cffi=1.11.5=py36he75722e_1
- chardet=3.0.4=py36_1
- click=7.0=py36_0
- cloudpickle=0.8.0=py36_0
- colorama=0.3.9=py36h489cec4_0
- configparser=3.7.3=py36_1
- cryptography=2.2.2=py36h14c3975_0
- cudnn=7.6.0=cuda10.0_0
- cupti=10.0.130=0
- cycler=0.10.0=py36_0
- cython=0.28.2=py36h14c3975_0
- decorator=4.3.0=py36_0
- docutils=0.14=py36_0
- entrypoints=0.2.3=py36_2
- et_xmlfile=1.0.1=py36hd6bccc3_0
- flask=1.0.2=py36_1
- freetype=2.8=hab7d2ae_1
- gast=0.2.2=py36_0
- gitdb2=2.0.5=py36_0
- gitpython=2.1.11=py36_0
- gmp=6.1.2=h6c8ec71_1
- grpcio=1.12.1=py36hdbcaa40_0
- gunicorn=19.9.0=py36_0
- h5py=2.8.0=py36h989c5e5_3
- hdf5=1.10.2=hba1933b_1
- html5lib=1.0.1=py36_0
- icu=58.2=h9c2bf20_1
- idna=2.6=py36h82fb2a8_1
- intel-openmp=2018.0.0=8
- ipython=6.4.0=py36_1
- ipython_genutils=0.2.0=py36hb52b0d5_0
- itsdangerous=0.24=py36_1
- jdcal=1.4=py36_0
- jedi=0.12.0=py36_1
- jinja2=2.10=py36_0
- jmespath=0.9.4=py_0
- jpeg=9b=h024ee3a_2
- jsonschema=2.6.0=py36_0
- jupyter_client=5.2.3=py36_0
- jupyter_core=4.4.0=py36_0
- keras=2.2.4=0
- keras-applications=1.0.8=py_0
- keras-base=2.2.4=py36_0
- keras-preprocessing=1.1.0=py_1
- krb5=1.16.1=hc83ff2d_6
- libedit=3.1.20170329=h6b74fdf_2
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.3.0=hdf63c60_0
- libgfortran-ng=7.2.0=hdf63c60_3
- libpng=1.6.34=hb9fc6fc_0
- libpq=10.4=h1ad7b7a_0
- libprotobuf=3.8.0=hd408876_0
- libsodium=1.0.16=h1bed415_0
- libstdcxx-ng=7.3.0=hdf63c60_0
- libtiff=4.0.9=he85c1e1_2
- libxgboost=0.90=h688424c_0
- libxml2=2.9.8=h26e45fe_1
- libxslt=1.1.32=h1312cb7_0
- llvmlite=0.23.1=py36hdbcaa40_0
- lxml=4.2.1=py36h23eabaa_0
- mako=1.0.10=py_0
- markdown=3.1.1=py36_0
- markupsafe=1.0=py36h14c3975_1
- mistune=0.8.3=py36h14c3975_1
- mkl=2019.4=243
- mkl_fft=1.0.12=py36ha843d7b_0
- mkl_random=1.0.2=py36hd81dba3_0
- mock=3.0.5=py36_0
- msgpack-python=0.5.6=py36h6bb024c_1
- nbconvert=5.3.1=py36_0
- nbformat=4.4.0=py36h31c9010_0
- ncurses=6.1=he6710b0_1
- ninja=1.9.0=py36hfd86e86_0
- numba=0.38.0=py36h637b7d7_0
- numpy=1.16.2=py36h7e9f1db_0
- numpy-base=1.16.2=py36hde5b4d6_0
- olefile=0.45.1=py36_0
- openpyxl=2.5.3=py36_0
- openssl=1.0.2o=h14c3975_1
- pandas=0.23.0=py36h637b7d7_0
- pandocfilters=1.4.2=py36_1
- paramiko=2.4.2=py36_0
- parso=0.2.0=py36_0
- pathlib2=2.3.2=py36_0
- patsy=0.5.0=py36_0
- pexpect=4.5.0=py36_0
- pickleshare=0.7.4=py36h63277f8_0
- pillow=5.1.0=py36h3deb7b8_0
- pip=10.0.1=py36_0
- ply=3.11=py36_0
- prompt_toolkit=1.0.15=py36_0
- protobuf=3.8.0=py36he6710b0_0
- psycopg2=2.7.5=py36hb7f436b_0
- ptyprocess=0.5.2=py36h69acd42_0
- py-xgboost=0.90=py36h688424c_0
- py-xgboost-gpu=0.90=py36h28bbb66_0
- pyasn1=0.4.5=py_0
- pycparser=2.18=py36_1
- pygments=2.2.0=py36_0
- pynacl=1.3.0=py36h7b6447c_0
- pyopenssl=18.0.0=py36_0
- pyparsing=2.2.0=py36_1
- pysocks=1.6.8=py36_0
- python=3.6.5=hc3d631a_2
- python-dateutil=2.7.3=py36_0
- python-editor=1.0.4=py_0
- pytz=2018.4=py36_0
- pyyaml=5.1=py36h7b6447c_0
- pyzmq=17.0.0=py36h14c3975_3
- readline=7.0=h7b6447c_5
- requests=2.18.4=py36he2e5f8d_1
- s3transfer=0.1.13=py36_0
- scikit-learn=0.20.3=py36hd81dba3_0
- scipy=1.1.0=py36h7c811a0_2
- setuptools=39.1.0=py36_0
- simplegeneric=0.8.1=py36_2
- simplejson=3.16.0=py36h14c3975_0
- singledispatch=3.4.0.3=py36h7a266c3_0
- six=1.11.0=py36_1
- smmap2=2.0.5=py36_0
- sqlite=3.23.1=he433501_0
- sqlparse=0.3.0=py_0
- statsmodels=0.9.0=py36h035aef0_0
- tabulate=0.8.3=py36_0
- tensorboard=1.13.1=py36hf484d3e_0
- tensorflow-estimator=1.13.0=py_0
- termcolor=1.1.0=py36_1
- testpath=0.3.1=py36_0
- tk=8.6.7=hc745277_3
- tornado=5.0.2=py36h14c3975_0
- traitlets=4.3.2=py36h674d592_0
- urllib3=1.22=py36hbe7ace6_0
- virtualenv=16.0.0=py36_0
- wcwidth=0.1.7=py36hdf4376a_0
- webencodings=0.5.1=py36_1
- werkzeug=0.14.1=py36_0
- wheel=0.31.1=py36_0
- wrapt=1.11.1=py36h7b6447c_0
- xz=5.2.4=h14c3975_4
- yaml=0.1.7=had09818_2
- zeromq=4.2.5=hf484d3e_1
- zlib=1.2.11=h7b6447c_3
- pytorch=1.1.0=py3.6_cuda10.0.130_cudnn7.5.1_0
- torchvision=0.3.0=py36_cu10.0.130_1
- pip:
- databricks-cli==0.8.7
- docker==4.0.2
- fusepy==2.0.4
- future==0.17.1
- horovod==0.16.4
- hyperopt==0.1.2.db6
- kiwisolver==1.1.0
- matplotlib==2.2.2
- mleap==0.8.1
- mlflow==1.0.0
- msgpack==0.5.6
- networkx==2.2
- nose==1.3.7
- nose-exclude==0.5.0
- psutil==5.6.3
- pyarrow==0.13.0
- pymongo==3.8.0
- querystring-parser==1.2.3
- seaborn==0.8.1
- tensorboardx==1.7
- tqdm==4.32.2
- websocket-client==0.56.0
prefix: /databricks/python3
Python 2 v clusterech procesoru
name: null
channels:
- pytorch
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- _py-xgboost-mutex=2.0=cpu_0
- _tflow_select=2.3.0=mkl
- absl-py=0.7.1=py27_0
- asn1crypto=0.24.0=py27_0
- astor=0.7.1=py27_0
- backports=1.0=py_2
- backports.shutil_get_terminal_size=1.0.0=py27_2
- backports.weakref=1.0.post1=py_1
- backports_abc=0.5=py_0
- bcrypt=3.1.6=py27h7b6447c_0
- blas=1.0=mkl
- bleach=2.1.3=py27_0
- boto=2.48.0=py27_1
- boto3=1.7.62=py27h28b3542_1
- botocore=1.10.62=py27h28b3542_0
- ca-certificates=2018.03.07=0
- certifi=2018.4.16=py27_0
- cffi=1.11.5=py27he75722e_1
- chardet=3.0.4=py27_1
- click=7.0=py27_0
- cloudpickle=0.8.0=py27_0
- colorama=0.3.9=py27h5cde069_0
- configparser=3.7.3=py27_1
- cryptography=2.2.2=py27h14c3975_0
- cycler=0.10.0=py27hc7354d3_0
- cython=0.28.2=py27h14c3975_0
- decorator=4.3.0=py27_0
- docutils=0.14=py27_0
- entrypoints=0.2.3=py27_2
- enum34=1.1.6=py27_1
- et_xmlfile=1.0.1=py27_0
- flask=1.0.2=py27_1
- freetype=2.8=hab7d2ae_1
- funcsigs=1.0.2=py27_0
- functools32=3.2.3.2=py27_1
- future=0.17.1=py27_0
- futures=3.2.0=py27_0
- gast=0.2.2=py27_0
- gitdb2=2.0.5=py27_0
- gitpython=2.1.11=py27_0
- gmp=6.1.2=h6c8ec71_1
- grpcio=1.12.1=py27hdbcaa40_0
- gunicorn=19.9.0=py27_0
- h5py=2.8.0=py27h989c5e5_3
- hdf5=1.10.2=hba1933b_1
- html5lib=1.0.1=py27_0
- icu=58.2=h9c2bf20_1
- idna=2.6=py27h5722d68_1
- intel-openmp=2018.0.0=8
- ipaddress=1.0.22=py27_0
- ipython=5.7.0=py27_0
- ipython_genutils=0.2.0=py27_0
- itsdangerous=0.24=py27_1
- jdcal=1.4=py27_0
- jinja2=2.10=py27_0
- jmespath=0.9.4=py_0
- jpeg=9b=h024ee3a_2
- jsonschema=2.6.0=py27h7ed5aa4_0
- jupyter_client=5.2.3=py27_0
- jupyter_core=4.4.0=py27_0
- keras=2.2.4=0
- keras-applications=1.0.8=py_0
- keras-base=2.2.4=py27_0
- keras-preprocessing=1.1.0=py_1
- krb5=1.16.1=hc83ff2d_6
- libedit=3.1.20170329=h6b74fdf_2
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.3.0=hdf63c60_0
- libgfortran-ng=7.2.0=hdf63c60_3
- libpng=1.6.34=hb9fc6fc_0
- libpq=10.4=h1ad7b7a_0
- libprotobuf=3.8.0=hd408876_0
- libsodium=1.0.16=h1bed415_0
- libstdcxx-ng=7.3.0=hdf63c60_0
- libtiff=4.0.9=he85c1e1_2
- libxgboost=0.90=he6710b0_0
- libxml2=2.9.8=h26e45fe_1
- libxslt=1.1.32=h1312cb7_0
- linecache2=1.0.0=py27_0
- llvmlite=0.23.1=py27hdbcaa40_0
- lxml=4.2.1=py27h23eabaa_0
- mako=1.0.10=py_0
- markdown=3.1.1=py27_0
- markupsafe=1.0=py27h14c3975_1
- mistune=0.8.3=py27h14c3975_1
- mkl=2019.4=243
- mkl_fft=1.0.12=py27ha843d7b_0
- mkl_random=1.0.2=py27hd81dba3_0
- mock=3.0.5=py27_0
- msgpack-python=0.5.6=py27h6bb024c_1
- nbconvert=5.3.1=py27_0
- nbformat=4.4.0=py27hed7f2b2_0
- ncurses=6.1=he6710b0_1
- ninja=1.9.0=py27hfd86e86_0
- numba=0.38.0=py27h637b7d7_0
- numpy=1.16.2=py27h7e9f1db_0
- numpy-base=1.16.2=py27hde5b4d6_0
- olefile=0.45.1=py27_0
- openpyxl=2.5.3=py27_0
- openssl=1.0.2o=h14c3975_1
- pandas=0.23.0=py27h637b7d7_0
- pandocfilters=1.4.2=py27_1
- paramiko=2.4.2=py27_0
- pathlib2=2.3.2=py27_0
- patsy=0.5.0=py27_0
- pexpect=4.5.0=py27_0
- pickleshare=0.7.4=py27_0
- pillow=5.1.0=py27h3deb7b8_0
- pip=10.0.1=py27_0
- ply=3.11=py27_0
- prompt_toolkit=1.0.15=py27_0
- protobuf=3.8.0=py27he6710b0_0
- psycopg2=2.7.5=py27hb7f436b_0
- ptyprocess=0.5.2=py27h4ccb14c_0
- py-xgboost=0.90=py27he6710b0_0
- py-xgboost-cpu=0.90=py27_0
- pyasn1=0.4.5=py_0
- pycparser=2.18=py27_1
- pygments=2.2.0=py27_0
- pynacl=1.3.0=py27h7b6447c_0
- pyopenssl=18.0.0=py27_0
- pyparsing=2.2.0=py27_1
- pysocks=1.6.8=py27_0
- python=2.7.15=h1571d57_0
- python-dateutil=2.7.3=py27_0
- python-editor=1.0.4=py_0
- pytz=2018.4=py27_0
- pyyaml=5.1=py27h7b6447c_0
- pyzmq=17.0.0=py27h14c3975_3
- readline=7.0=h7b6447c_5
- requests=2.18.4=py27hc5b0589_1
- s3transfer=0.1.13=py27_0
- scandir=1.7=py27h14c3975_0
- scikit-learn=0.20.3=py27hd81dba3_0
- scipy=1.1.0=py27h7c811a0_2
- setuptools=39.1.0=py27_0
- simplegeneric=0.8.1=py27_2
- simplejson=3.16.0=py27h14c3975_0
- singledispatch=3.4.0.3=py27_0
- six=1.11.0=py27_1
- smmap2=2.0.5=py27_0
- sqlite=3.23.1=he433501_0
- sqlparse=0.3.0=py_0
- statsmodels=0.9.0=py27h035aef0_0
- tabulate=0.8.3=py27_0
- tensorboard=1.13.1=py27hf484d3e_0
- tensorflow=1.13.1=mkl_py27h74ee40f_0
- tensorflow-base=1.13.1=mkl_py27h7ce6ba3_0
- tensorflow-estimator=1.13.0=py_0
- tensorflow-mkl=1.13.1=h4fcabd2_0
- termcolor=1.1.0=py27_1
- testpath=0.3.1=py27hc38d2c4_0
- tk=8.6.7=hc745277_3
- tornado=5.0.2=py27h14c3975_0
- traceback2=1.4.0=py27_0
- traitlets=4.3.2=py27_0
- unittest2=1.1.0=py27_0
- urllib3=1.22=py27ha55213b_0
- virtualenv=16.0.0=py27_0
- wcwidth=0.1.7=py27h9e3e1ab_0
- webencodings=0.5.1=py27_1
- werkzeug=0.14.1=py27_0
- wheel=0.31.1=py27_0
- wrapt=1.11.1=py27h7b6447c_0
- xz=5.2.4=h14c3975_4
- yaml=0.1.7=had09818_2
- zeromq=4.2.5=hf484d3e_1
- zlib=1.2.11=h7b6447c_3
- pytorch-cpu=1.1.0=py2.7_cpu_0
- torchvision-cpu=0.3.0=py27_cuNone_1
- pip:
- backports.functools-lru-cache==1.5
- backports.ssl-match-hostname==3.7.0.1
- databricks-cli==0.8.7
- docker==4.0.2
- fusepy==2.0.4
- horovod==0.16.4
- hyperopt==0.1.2.db6
- kiwisolver==1.1.0
- matplotlib==2.2.2
- mleap==0.8.1
- mlflow==1.0.0
- msgpack==0.5.6
- networkx==2.2
- nose==1.3.7
- nose-exclude==0.5.0
- psutil==5.6.3
- pyarrow==0.13.0
- pymongo==3.8.0
- querystring-parser==1.2.3
- seaborn==0.8.1
- subprocess32==3.5.4
- tensorboardx==1.7
- torchvision==0.3.0
- tqdm==4.32.2
- websocket-client==0.56.0
prefix: /databricks/python2
Python 2 v clusterech GPU
name: null
channels:
- Databricks
- pytorch
- defaults
dependencies:
- tensorflow=1.13.1.db1=gpu_py27h8e347d7_0
- tensorflow-base=1.13.1.db1=gpu_py27he292aa2_0
- tensorflow-gpu=1.13.1.db1=h0d30ee6_0
- _libgcc_mutex=0.1=main
- _py-xgboost-mutex=1.0=gpu_0
- _tflow_select=2.1.0=gpu
- absl-py=0.7.1=py27_0
- asn1crypto=0.24.0=py27_0
- astor=0.7.1=py27_0
- backports=1.0=py_2
- backports.shutil_get_terminal_size=1.0.0=py27_2
- backports.weakref=1.0.post1=py_1
- backports_abc=0.5=py_0
- bcrypt=3.1.6=py27h7b6447c_0
- blas=1.0=mkl
- bleach=2.1.3=py27_0
- boto=2.48.0=py27_1
- boto3=1.7.62=py27h28b3542_1
- botocore=1.10.62=py27h28b3542_0
- ca-certificates=2018.03.07=0
- certifi=2018.4.16=py27_0
- cffi=1.11.5=py27he75722e_1
- chardet=3.0.4=py27_1
- click=7.0=py27_0
- cloudpickle=0.8.0=py27_0
- colorama=0.3.9=py27_0
- configparser=3.7.3=py27_1
- cryptography=2.2.2=py27h14c3975_0
- cudnn=7.6.0=cuda10.0_0
- cupti=10.0.130=0
- cycler=0.10.0=py27_0
- cython=0.28.2=py27h14c3975_0
- decorator=4.3.0=py27_0
- docutils=0.14=py27hae222c1_0
- entrypoints=0.2.3=py27_2
- enum34=1.1.6=py27_1
- et_xmlfile=1.0.1=py27h75840f5_0
- flask=1.0.2=py27_1
- freetype=2.8=hab7d2ae_1
- funcsigs=1.0.2=py27_0
- functools32=3.2.3.2=py27_1
- future=0.17.1=py27_0
- futures=3.2.0=py27_0
- gast=0.2.2=py27_0
- gitdb2=2.0.5=py27_0
- gitpython=2.1.11=py27_0
- gmp=6.1.2=h6c8ec71_1
- grpcio=1.12.1=py27hdbcaa40_0
- gunicorn=19.9.0=py27_0
- h5py=2.8.0=py27h989c5e5_3
- hdf5=1.10.2=hba1933b_1
- html5lib=1.0.1=py27_0
- icu=58.2=h9c2bf20_1
- idna=2.6=py27h5722d68_1
- intel-openmp=2018.0.0=8
- ipaddress=1.0.22=py27_0
- ipython=5.7.0=py27_0
- ipython_genutils=0.2.0=py27h89fb69b_0
- itsdangerous=0.24=py27_1
- jdcal=1.4=py27_0
- jinja2=2.10=py27_0
- jmespath=0.9.4=py_0
- jpeg=9b=h024ee3a_2
- jsonschema=2.6.0=py27h7ed5aa4_0
- jupyter_client=5.2.3=py27_0
- jupyter_core=4.4.0=py27_0
- keras=2.2.4=0
- keras-applications=1.0.8=py_0
- keras-base=2.2.4=py27_0
- keras-preprocessing=1.1.0=py_1
- krb5=1.16.1=hc83ff2d_6
- libedit=3.1.20170329=h6b74fdf_2
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.3.0=hdf63c60_0
- libgfortran-ng=7.2.0=hdf63c60_3
- libpng=1.6.34=hb9fc6fc_0
- libpq=10.4=h1ad7b7a_0
- libprotobuf=3.8.0=hd408876_0
- libsodium=1.0.16=h1bed415_0
- libstdcxx-ng=7.3.0=hdf63c60_0
- libtiff=4.0.9=he85c1e1_2
- libxgboost=0.90=h688424c_0
- libxml2=2.9.8=h26e45fe_1
- libxslt=1.1.32=h1312cb7_0
- linecache2=1.0.0=py27_0
- llvmlite=0.23.1=py27hdbcaa40_0
- lxml=4.2.1=py27h23eabaa_0
- mako=1.0.10=py_0
- markdown=3.1.1=py27_0
- markupsafe=1.0=py27h14c3975_1
- mistune=0.8.3=py27h14c3975_1
- mkl=2019.4=243
- mkl_fft=1.0.12=py27ha843d7b_0
- mkl_random=1.0.2=py27hd81dba3_0
- mock=3.0.5=py27_0
- msgpack-python=0.5.6=py27h6bb024c_1
- nbconvert=5.3.1=py27_0
- nbformat=4.4.0=py27hed7f2b2_0
- ncurses=6.1=he6710b0_1
- ninja=1.9.0=py27hfd86e86_0
- numba=0.38.0=py27h637b7d7_0
- numpy=1.16.2=py27h7e9f1db_0
- numpy-base=1.16.2=py27hde5b4d6_0
- olefile=0.45.1=py27_0
- openpyxl=2.5.3=py27_0
- openssl=1.0.2o=h14c3975_1
- pandas=0.23.0=py27h637b7d7_0
- pandocfilters=1.4.2=py27_1
- paramiko=2.4.2=py27_0
- pathlib2=2.3.2=py27_0
- patsy=0.5.0=py27_0
- pexpect=4.5.0=py27_0
- pickleshare=0.7.4=py27h09770e1_0
- pillow=5.1.0=py27h3deb7b8_0
- pip=10.0.1=py27_0
- ply=3.11=py27_0
- prompt_toolkit=1.0.15=py27_0
- protobuf=3.8.0=py27he6710b0_0
- psycopg2=2.7.5=py27hb7f436b_0
- ptyprocess=0.5.2=py27h4ccb14c_0
- py-xgboost=0.90=py27h688424c_0
- py-xgboost-gpu=0.90=py27h28bbb66_0
- pyasn1=0.4.5=py_0
- pycparser=2.18=py27_1
- pygments=2.2.0=py27_0
- pynacl=1.3.0=py27h7b6447c_0
- pyopenssl=18.0.0=py27_0
- pyparsing=2.2.0=py27_1
- pysocks=1.6.8=py27_0
- python=2.7.15=h1571d57_0
- python-dateutil=2.7.3=py27_0
- python-editor=1.0.4=py_0
- pytz=2018.4=py27_0
- pyyaml=5.1=py27h7b6447c_0
- pyzmq=17.0.0=py27h14c3975_3
- readline=7.0=h7b6447c_5
- requests=2.18.4=py27hc5b0589_1
- s3transfer=0.1.13=py27_0
- scandir=1.7=py27h14c3975_0
- scikit-learn=0.20.3=py27hd81dba3_0
- scipy=1.1.0=py27h7c811a0_2
- setuptools=39.1.0=py27_0
- simplegeneric=0.8.1=py27_2
- simplejson=3.16.0=py27h14c3975_0
- singledispatch=3.4.0.3=py27h9bcb476_0
- six=1.11.0=py27_1
- smmap2=2.0.5=py27_0
- sqlite=3.23.1=he433501_0
- sqlparse=0.3.0=py_0
- statsmodels=0.9.0=py27h035aef0_0
- tabulate=0.8.3=py27_0
- tensorboard=1.13.1=py27hf484d3e_0
- tensorflow-estimator=1.13.0=py_0
- termcolor=1.1.0=py27_1
- testpath=0.3.1=py27_0
- tk=8.6.7=hc745277_3
- tornado=5.0.2=py27h14c3975_0
- traceback2=1.4.0=py27_0
- traitlets=4.3.2=py27hd6ce930_0
- unittest2=1.1.0=py27_0
- urllib3=1.22=py27ha55213b_0
- virtualenv=16.0.0=py27_0
- wcwidth=0.1.7=py27_0
- webencodings=0.5.1=py27_1
- werkzeug=0.14.1=py27_0
- wheel=0.31.1=py27_0
- wrapt=1.11.1=py27h7b6447c_0
- xz=5.2.4=h14c3975_4
- yaml=0.1.7=had09818_2
- zeromq=4.2.5=hf484d3e_1
- zlib=1.2.11=h7b6447c_3
- pytorch=1.1.0=py2.7_cuda10.0.130_cudnn7.5.1_0
- torchvision=0.3.0=py27_cu10.0.130_1
- pip:
- backports.functools-lru-cache==1.5
- backports.ssl-match-hostname==3.7.0.1
- databricks-cli==0.8.7
- docker==4.0.2
- fusepy==2.0.4
- horovod==0.16.4
- hyperopt==0.1.2.db6
- kiwisolver==1.1.0
- matplotlib==2.2.2
- mleap==0.8.1
- mlflow==1.0.0
- msgpack==0.5.6
- networkx==2.2
- nose==1.3.7
- nose-exclude==0.5.0
- psutil==5.6.3
- pyarrow==0.13.0
- pymongo==3.8.0
- querystring-parser==1.2.3
- seaborn==0.8.1
- subprocess32==3.5.4
- tensorboardx==1.7
- tqdm==4.32.2
- websocket-client==0.56.0
prefix: /databricks/python2
Balíčky Spark obsahující moduly Pythonu
Balíček Spark | Modul Pythonu | Verze |
---|---|---|
graphframes | graphframes | 0.7.0-db1-spark2.4 |
spark-deep-learning | Sparkdl | 1.5.0-db4-spark2.4 |
tensorframes | tensorframes | 0.7.0-s_2.11 |
Knihovny jazyka R
Knihovny jazyka R jsou identické s knihovnami jazyka R v Databricks Runtime 5.5.
Knihovny Java a Scala (cluster Scala 2.11)
Kromě knihoven Java a Scala v Databricks Runtime 5.5 obsahuje Databricks Runtime 5.5 LTS pro machine Učení následující jary:
ID skupiny | ID artefaktu | Verze |
---|---|---|
com.databricks | spark-deep-learning | 1.5.0-db4-spark2.4 |
com.typesafe.akka | akka-actor_2.11 | 2.3.11 |
ml.combust.mleap | mleap-databricks-runtime_2.11 | 0.13.0 |
ml.dmlc | xgboost4j | 0.90 |
ml.dmlc | xgboost4j-spark | 0.90 |
org.graphframes | graphframes_2.11 | 0.7.0-db1-spark2.4 |
org.tensorflow | libtensorflow | 1.13.1 |
org.tensorflow | libtensorflow_jni | 1.13.1 |
org.tensorflow | spark-tensorflow-connector_2.11 | 1.13.1 |
org.tensorflow | tensorflow | 1.13.1 |
org.tensorframes | tensorframes | 0.7.0-s_2.11 |