Azure ML create data asset from MLTable
I'd like to add data assets via Azure ML UI. Therefore I created an MLTable file next to my data:
\---
type: mltable
paths:
-
pattern: ./*/*.parquet
transformations:
-
read_parquet:
include_path_column: false
-
convert_column_types:
-
columns:
pattern: "*_date"
to_type:
datetime:
formats:
- "%Y-%m-%d %H:%M:%S"
I would assume this format sticks to the schema defined here: https://azuremlschemasprod.azureedge.net/latest/MLTable.schema.json
Although I get the following information when selecting the folder with the above mentioned MLTable file:
Invalid MLTable format, convert_column_types does not support some of the provided properties: to_type. If you want to proceed, consider checking "Skip data validation" and click "Next." We will not validate your data path or query, or try to access your data for preview and schema checks.
Any ideas how to fix this?
The section of the schema I am relating to is this:
...
{
"type": "object",
"properties": {
"columns": {
"oneOf": [
{
"type": "string",
"description": "single column to be selected"
},
{
"type": "array",
"description": "list of columns to be selected",
"items": {
"type": "string"
}
},
{
"type": "object",
"description": "pattern and properties defining columns to be selected",
"properties": {
"pattern": {
"type": "string"
},
"ignore_case": {
"type": "boolean",
"default": false
},
"invert": {
"type": "boolean",
"default": false
}
}
}
]
},
"to_type": {
"type": "object",
"properties": {
"datetime": {
"type": "object",
"properties": {
"formats": {
"type": "array",
"items": {
"type": "string"
}
}
},
"additionalProperties": false
}
},
"additionalProperties": false
}
},
"additionalProperties": false
},
...