This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 9%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKA%3C/text%3E%3C/svg%3E)
I have created a custom pipeline in Azure Machine Learning that involves executing a script called data_prep.py. This script requires the en_core_web_sm model from the Spacy library for data cleaning purposes.
I have defined an environment YAML file for my pipeline, where I included the spacy package as a dependency. However, when I try to include the en_core_web_sm model in the YAML file, it fails to create the environment.
I have also attempted to download the en_core_web_sm model manually and store it in a folder, and then referenced the model path in my data_prep.py script. However, this approach is not working either.
I would greatly appreciate any guidance on how to correctly include the en_core_web_sm model in my Azure ML pipeline environment. Is there a recommended method to include Spacy models in Azure ML environments? How can I ensure that the en_core_web_sm model is accessible to my pipeline and can be utilized by data_prep.py?
Thank you for your assistance.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 1%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYM%3C/text%3E%3C/svg%3E)
Hello [@Kavinamoole, Abhishek]/users/na/?userid=310be9bd-66aa-4ab9-8091-dba14acc54e0)
Thanks for reaching out to us, to include the en_core_web_sm model in your Azure ML pipeline environment, you can try the following steps:
Install the en_core_web_sm model in your environment: You can install the en_core_web_sm model in your environment by adding it as a dependency in your environment YAML file. Here is an example of how to add the en_core_web_sm model as a dependency:
dependencies:
- python=3.8
- spacy
- pip:
- https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.1.0/en_core_web_sm-3.1.0-py3-none-any.whl
This will install the en_core_web_sm model from the official Spacy models repository.
Load the en_core_web_sm model in your script: Once the en_core_web_sm model is installed in your environment, you can load it in your script using the spacy.load() function. Here is an example of how to load the en_core_web_sm model in your script:
import spacy
nlp = spacy.load("en_core_web_sm")
This will load the en_core_web_sm model into memory and allow you to use it for data cleaning purposes.
Verify that the model is accessible: To verify that the en_core_web_sm model is accessible in your environment, you can run a simple test script that loads the model and performs some basic text processing tasks. If the script runs without errors, then the en_core_web_sm model is accessible and can be used in your pipeline.
I hope this helps! Please have a try and let me know if you have any other questions.
Regards,
Yutong
Due to restrictions imposed by my organization, Azure is unable to connect to the internet and download from GitHub. I attempted the previously suggested method, but encountered an SSL error. Are there any alternative approaches available? Could you please provide guidance on how to proceed?