Can I train models in Azure ML by uploading local files instead of using Azure Blob Storage? What are the limitations?

Question

I'm learning Azure Machine Learning and had a question about data ingestion.

Instead of first uploading my dataset to an Azure Blob Storage account (or registering it as a datastore), can I simply upload the dataset directly from my local computer into Azure ML and use it for model training?

My main reason is to reduce Azure Storage costs while experimenting with small datasets.

If this approach is possible:

What are the limitations compared to using Blob Storage?

Is the file still stored somewhere in Azure behind the scenes?

Are there any restrictions on dataset size, reproducibility, or collaboration?

Would this approach work only for experimentation, or is it also suitable for production workloads?

What is Microsoft's recommended practice for training datasets in Azure ML?

I'd appreciate any clarification from people who have used both approaches.

Answer 1

Yes. Azure Machine Learning supports creating a data asset from local files. That creates a reference to the data source location together with metadata, and for smaller uploads this is a supported way to move data from a local machine into cloud storage for use in notebooks and jobs.

Key limitations and tradeoffs compared to using storage directly:

1. The data is still stored in Azure
   • Uploading local files through Azure Machine Learning does not avoid cloud storage.
   • The local file is uploaded to a designated cloud storage resource when the data asset is created.
   • In studio workflows, the default destination is typically the workspace default datastore, such as workspaceblobstore.
2. Best suited for smaller datasets
   • The guidance explicitly says data asset creation works well for smaller uploads from a local machine.
   • For larger uploads, a dedicated transfer tool such as AzCopy is recommended.
3. Versioning matters for reproducibility
   • Each data asset version must be unique.
   • If a version already exists, creation fails unless a new version is used.
   • Omitting the version lets Azure Machine Learning generate one automatically.
   • This versioning helps reproducibility more than ad hoc local uploads without managed versions.
4. Collaboration is better with workspace-managed assets
   • A workspace is designed for collaboration, and users can share versioned assets and storage references.
   • Using data assets is more suitable for shared team workflows than relying on unmanaged local-only copies.
5. Production suitability
   • Local upload through data asset creation is practical for experimentation and small datasets.
   • For larger or more operational workloads, the documented pattern is to use cloud storage locations, datastores, and data assets rather than treating local upload as the primary production ingestion path.
6. Cost expectations
   • Creating a data asset creates a reference plus metadata, and the data remains in its existing location without extra storage cost when the source is already in cloud storage.
   • That benefit does not mean local-file upload is storage-free. When starting from a local machine, the file is uploaded to Azure storage.

Recommended practice:

• For experimentation with small datasets: uploading local files as an Azure Machine Learning data asset is appropriate.
• For repeatable, collaborative, or larger-scale training: use Azure storage, register it through a datastore when needed, and manage the training input as versioned data assets.
• If using Automated ML in studio, the documented flow is to upload the file to the workspace default datastore and use it as a Tabular data asset.

A concise answer to each question:

• Can local files be used for training? Yes, by creating a data asset from local files.
• What are the limitations? Mainly scale and operational maturity; smaller uploads are fine, larger uploads should use dedicated tools and cloud storage patterns.
• Is the file stored in Azure behind the scenes? Yes, the local file is uploaded to cloud storage.
• Restrictions on size, reproducibility, collaboration? Small uploads are the intended fit; versioned data assets improve reproducibility; workspace assets support collaboration.
• Experimentation or production? Good for experimentation and small datasets; cloud storage plus datastores/data assets is the better production pattern.
• Recommended Microsoft practice? Use data assets and cloud storage references, with datastores for existing Azure storage and versioned assets for repeatable ML workflows.

---

References:

• Tutorial: Upload, access, and explore your data in Azure Machine Learning
• Data concepts in Azure Machine Learning
• What is Azure Machine Learning?
• Tutorial: Train a classification model with no-code AutoML in the Azure Machine Learning studio