This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 79%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPS%3C/text%3E%3C/svg%3E)
we need an end-to-end python code for creating MLTable data asset which will be used an input for azure autoML pipeline. currently we are facing issue creating the said data asset from python sdk v2(azure-ai-ml) and reading it during autoML run. We are not able figure if the issue is with data format or way we are creating the data asset. Please help us resolve the issue.
Thank you
maybe the following code snippets can help you
Upload your data to Azure Blob Storage
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
try:
blob_service_client = BlobServiceClient.from_connection_string(my_connection_string)
blob_client = blob_service_client.get_blob_client("mycontainer", "mysamplefile")
with open("./SampleSource.txt", "rb") as data:
blob_client.upload_blob(data)
except Exception as ex:
print('Exception:')
print(ex)
Create a Dataset in Azure Machine Learning
from azureml.core import Workspace, Datastore, Dataset
# get existing workspace
workspace = Workspace.from_config()
# get datastore associated with the workspace
datastore = Datastore.get(workspace, 'my_datastore')
# create tabular dataset from the data in Blob storage
dataset = Dataset.Tabular.from_delimited_files(path=(datastore, 'mysamplefile'))
# register the tabular dataset
dataset = dataset.register(workspace=workspace,
name='my_dataset',
description='my description')
Use the Dataset in AutoML
from azureml.train.automl import AutoMLConfig
automl_config = AutoMLConfig(task='classification',
primary_metric='accuracy',
training_data=dataset,
label_column_name='my_label',
n_cross_validations=5)
automl_run = experiment.submit(automl_config)
I appreciate your help @Sedat SALMAN but we are trying to use azure-ai-ml package as i am using mlclient to handle all the resources. Do u recommend me to change package to azureml ?
azureml and azure-ai-ml packages are incompatible.
Hi @Ramr-msft , we are still facing issues during MLTable data consumption during sdk-driven automl run. Below are the images of the said errors. Any help would be appreciated. Thank you
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 11%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERM%3C/text%3E%3C/svg%3E)
@Pawan Singh Bisht Thanks for the details. It's dataset issue on your end. We are able to create ML table from azure storage Successfully. Here's an example of how you can create an MLTable data asset using the Python SDK v2 and use it as input for an Azure AutoML pipeline:
from azureml.core import Workspace, Dataset
from azureml.core.compute import AmlCompute
from azureml.core.compute_target import ComputeTargetException
from azureml.pipeline.core import Pipeline, PipelineData
from azureml.pipeline.steps import PythonScriptStep
from azureml.train.automl import AutoMLConfig
# Load the workspace from the configuration file
ws = Workspace.from_config()
# Define the path to the CSV file in Azure Blob Storage
data_path = 'azure_blob://<container_name>/<blob_path>'
# Create a TabularDatasetFactory object from the CSV file
data_factory = Dataset.Tabular.from_delimited_files(data_path)
# Convert the TabularDatasetFactory object to an MLTable object
ml_table = data_factory.to_table()
# Register the MLTable object as a dataset in the workspace
dataset_name = 'my_dataset'
dataset = ml_table.register(workspace=ws, name=dataset_name)
# Define the compute target for the AutoML run
compute_target_name = 'my_compute_target'
try:
compute_target = AmlCompute(workspace=ws, name=compute_target_name)
print('Found existing compute target.')
except ComputeTargetException:
print('Creating new compute target.')
compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_DS12_V2', max_nodes=4)
compute_target = AmlCompute.create(workspace=ws, name=compute_target_name, provisioning_configuration=compute_config)
compute_target.wait_for_completion(show_output=True)
# Define the output directory for the AutoML run
output_dir = PipelineData('output_dir', datastore=ws.get_default_datastore())
# Define the AutoML configuration
automl_config = AutoMLConfig(task='classification', primary_metric='accuracy', training_data=dataset, label_column_name='label', compute_target=compute_target, path=output_dir)
# Define the PythonScriptStep for running the AutoML pipeline
step_name = 'automl_step'
script_name = 'automl.py'
arguments = ['--output_dir', output_dir]
automl_step = PythonScriptStep(name=step_name, script_name=script_name, arguments=arguments, compute_target
@Ramr-msft thank you for your help. Were able to resolve issue with python SDK v2 only, took help from the Azure support. But ddnt azure fixed packages as python sdk versions? azureml is for python sdk v1 and azure-ai-ml for sdk v2.
@Pawan Singh Bisht No, Thanks for the details. Could you please help with the error details that you are facing. These Azure.ai.ml are used in the MLClient to get a handle to the required Azure Machine Learning workspace. Here are the samples.
https://github.com/Azure/azureml-examples/tree/main/sdk/python/using-mltable/from-paths-example