Synapse Analytics Requirements.txt not loaded

Ana Del Campo 121 Reputation points Microsoft Employee
2020-10-23T13:52:04.79+00:00

I have been trying to install google-api's libraries to a notebook, I need to extract google classroom data into synapse. I uploaded the requirements.txt file - even created a new cluster. Not the libraries are not being loaded, even after re-starting the cluster. I have reviewed other posts with similar questions, followed the instructions but still cannot install the libraries. Here is the entire requirements file - did a pip freeze and added the needed libraries at the bottom:

absl-py==0.8.1
adal==1.2.2
alabaster==0.7.10
altair==3.2.0
applicationinsights==0.11.9
asn1crypto==1.0.1
astor==0.8.0
astroid==1.4.9
astropy==1.3.2
attrs==19.2.0
azure-common==1.1.25
azure-core==1.8.1
azure-graphrbac==0.61.1
azure-identity==1.2.0
azure-mgmt-authorization==0.60.0
azure-mgmt-containerregistry==2.8.0
azure-mgmt-keyvault==2.0.0
azure-mgmt-resource==5.1.0
azure-mgmt-storage==4.2.0
azure-storage-blob==2.1.0
azure-storage-common==2.1.0
azureml-automl-core==1.10.0.post1
azureml-automl-runtime==1.10.0.post1
azureml-core==1.10.0.post1
azureml-dataprep==1.10.1
azureml-dataprep-native==14.2.1
azureml-dataset-runtime==1.10.0
azureml-defaults==1.10.0
azureml-explain-model==1.10.0
azureml-interpret==1.10.0
azureml-model-management-sdk==1.0.1b6.post1
azureml-opendatasets==1.10.0
azureml-pipeline==1.10.0
azureml-pipeline-core==1.10.0
azureml-pipeline-steps==1.10.0
azureml-sdk==1.10.0
azureml-telemetry==1.10.0
azureml-train==1.10.0
azureml-train-automl==1.10.0
azureml-train-automl-client==1.10.0
azureml-train-automl-runtime==1.10.0
azureml-train-core==1.10.0
azureml-train-restclients-hyperdrive==1.10.0
Babel==2.4.0
backcall==0.2.0
backports.shutil-get-terminal-size==1.0.0
backports.tempfile==1.0
backports.weakref==1.0.post1
beautifulsoup4==4.6.0
bitarray==0.8.1
blaze==0.10.1
bleach==1.5.0
bokeh==1.3.4
boto==2.49.0
boto3==1.9.247
botocore==1.12.247
Bottleneck==1.2.1
certifi==2019.9.11
cffi==1.12.3
chardet==3.0.4
chart-studio==1.0.0
click==6.7
cloudpickle==1.2.2
clyent==1.2.2
colorama==0.3.9
configparser==3.7.4
contextlib2==0.6.0.post1
cryptography==2.7
cycler==0.10.0
Cython==0.29.13
cytoolz==0.8.2
dask==0.14.3
datashape==0.5.4
decorator==4.4.0
dill==0.3.1.1
distributed==1.16.3
distro==1.4.0
docker==4.1.0
docutils==0.15.2
dotnetcore2==2.1.14
entrypoints==0.3
et-xmlfile==1.0.1
fastcache==1.0.2
fire==0.2.1
Flask==1.0.3
Flask-Cors==3.0.2
fsspec==0.5.2
fusepy==3.0.1
gast==0.3.2
gensim==3.8.1
gevent==1.2.1
google-pasta==0.1.7
greenlet==0.4.12
grpcio==1.24.1
gunicorn==19.9.0
h5py==2.10.0
HeapDict==1.0.1
html5lib==0.999
idna==2.8
imageio==2.6.1
imagesize==0.7.1
importlib-metadata==0.23
interpret-community==0.14.4
interpret-core==0.2.1
ipykernel==4.6.1
ipython==7.8.0
ipython-genutils==0.2.0
ipywidgets==6.0.0
isodate==0.6.0
isort==4.2.5
itsdangerous==0.24
jdcal==1.3
jedi==0.10.2
jeepney==0.4.1
Jinja2==2.10.3
jmespath==0.9.4
joblib==0.14.1
json-logging-py==0.2
JsonForm==0.0.2
jsonpickle==1.2
jsonschema==3.1.1
JsonSir==0.0.2
jupyter==1.0.0
jupyter-client==5.0.1
jupyter-console==5.1.0
jupyter-core==4.3.0
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
keras2onnx==1.5.2
kiwisolver==1.1.0
lazy-object-proxy==1.2.2
liac-arff==2.5.0
library-metadata-cooker==0.0.1
lightgbm==2.2.3
llvmlite==0.18.0
locket==0.2.0
lxml==3.7.3
Markdown==3.1.1
MarkupSafe==1.1.1
matplotlib==3.1.1
missingno==0.4.2
mistune==0.7.4
mmlspark==1.0.0.dev1
more-itertools==7.2.0
mpmath==0.19
msal==1.5.0
msal-extensions==0.1.3
msgpack==0.6.2
msgpack-python==0.4.8
msrest==0.6.10
msrestazure==0.6.2
multimethods==1.0.0
multipledispatch==0.4.9
mypy==0.780
mypy-extensions==0.4.3
navigator-updater==0.1.0
nbconvert==5.1.1
nbformat==4.3.0
ndg-httpsclient==0.5.1
networkx==2.3
nimbusml==1.7.1
nltk==3.2.3
nose==1.3.7
notebook==5.0.0
notebookutils==2.4.4-20200918.3
numba==0.33.0
numexpr==2.6.2
numpy==1.16.2
numpydoc==0.6.0
oauthlib==3.1.0
odo==0.5.0
olefile==0.44
onnx==1.6.0
onnxconverter-common==1.6.0
onnxmltools==1.4.1
onnxruntime==1.0.0
openpyxl==2.4.7
packaging==19.2
pandas==0.23.4
pandocfilters==1.4.1
param==1.9.2
partd==1.0.0
pathlib2==2.2.1
pathspec==0.6.0
patsy==0.5.1
pep8==1.7.0
pexpect==4.2.1
pickleshare==0.7.4
Pillow==6.2.0
pip==9.0.1
plotly==4.1.1
ply==3.10
pmdarima==1.1.1
portalocker==1.7.1
prompt-toolkit==2.0.10
protobuf==3.10.0
psutil==5.2.2
ptyprocess==0.5.1
py==1.4.33
py-cpuinfo==5.0.0
py4j==0.10.7
pyarrow==0.15.1
pyasn1==0.4.7
pycosat==0.6.2
pycparser==2.19
pycrypto==2.6.1
pyct==0.4.6
pycurl==7.43.0
pyflakes==1.5.0
pygal==2.4.0
Pygments==2.2.0
PyJWT==1.7.1
pylint==1.6.4
pymssql==2.1.4
pyodbc==4.0.16
pyOpenSSL==19.0.0
pyparsing==2.4.2
pyrsistent==0.15.4
pyspark==2.4.4
pytest==3.0.7
python-dateutil==2.8.0
Python-EasyConfig==0.1.7
pytz==2019.3
PyWavelets==1.0.3
PyYAML==5.1.2
pyzmq==16.0.2
QtAwesome==0.4.4
qtconsole==4.3.0
QtPy==1.2.1
requests==2.22.0
requests-oauthlib==1.2.0
Resource==0.2.1
retrying==1.3.3
rope-py3k==0.9.4.post1
ruamel.yaml==0.16.10
ruamel.yaml.clib==0.2.2
s3transfer==0.2.1
scikit-image==0.15.0
scikit-learn==0.20.3
scipy==1.2.0
seaborn==0.9.0
SecretStorage==3.1.1
setuptools==41.4.0
shap==0.34.0
simplegeneric==0.8.1
singledispatch==3.4.0.3
six==1.12.0
skl2onnx==1.4.9
sklearn-pandas==1.7.0
smart-open==1.8.4
snowballstemmer==1.2.1
sortedcollections==0.5.3
sortedcontainers==2.1.0
Sphinx==1.5.6
spyder==3.1.4
SQLAlchemy==1.1.9
statsmodels==0.10.1
sympy==1.0
tables==3.3.0
tblib==1.4.0
tensorboard==1.14.0
tensorflow==1.14.0
tensorflow-estimator==1.14.0
termcolor==1.1.0
terminado==0.6
testpath==0.3
toolz==0.10.0
torch==1.4.0
tornado==6.0.3
tqdm==4.50.0
traitlets==4.3.2
typed-ast==1.4.1
typing-extensions==3.7.4
unicodecsv==0.14.1
urllib3==1.25.6
vega-datasets==0.7.0
wcwidth==0.1.7
websocket-client==0.56.0
Werkzeug==0.16.1
wheel==0.30.0
widgetsnbextension==2.0.0
wrapt==1.11.2
xgboost==0.90
xlrd==1.0.0
XlsxWriter==0.9.6
xlwt==1.2.0
zict==1.0.0
zipp==0.6.0
google-api-core==1.23.0
google-api-python-client==1.12.4
google-auth==1.22.1
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.1
googleapis-common-protos==1.52.0
httplib2==0.18.1

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,365 questions
0 comments No comments
{count} votes

Accepted answer
  1. HimanshuSinha-msft 19,376 Reputation points Microsoft Employee
    2020-10-28T18:37:28.107+00:00

    Thanks @Ana Del Campo ,

    As we connected offline and found that the filesystem under that the ADLS gen2 was mssing and adding that fixed the issue .

    Thanks Himanshu
    Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members


2 additional answers

Sort by: Most helpful
  1. HimanshuSinha-msft 19,376 Reputation points Microsoft Employee
    2020-10-23T19:06:48.777+00:00

    Hello @Ana Del Campo ,

    Thanks for the ask and also using the Microsoft Q&A.
    Since you never mentioned the library so I am notsure , but i do see the below installed as per your thread . Please scroll down till the end .

    google-api-core==1.23.0
    google-api-python-client==1.12.4
    google-auth==1.22.1
    google-auth-httplib2==0.0.4
    google-auth-oauthlib==0.4.1
    googleapis-common-protos==1.52.0*I

    Thanks Himanshu
    Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members


  2. Ana Del Campo 121 Reputation points Microsoft Employee
    2020-10-24T03:34:54.237+00:00

    That is weird because I had already granted those permissions to both myself and the synapse workspace, I tried it again for good measure but it did not work:

    34791-image.png

    0 comments No comments