Azure ML create data asset from MLTable

Schoenenberg, Martin 1 Reputation point
2022-10-19T08:58:34.803+00:00

I'd like to add data assets via Azure ML UI. Therefore I created an MLTable file next to my data:

   \---   
   type: mltable  
   paths:   
     -   
       pattern: ./*/*.parquet  
   transformations:   
     -   
       read_parquet:   
         include_path_column: false  
     -   
       convert_column_types:   
         -   
           columns:   
             pattern: "*_date"  
           to_type:   
             datetime:   
               formats:   
                 - "%Y-%m-%d %H:%M:%S"  

I would assume this format sticks to the schema defined here: https://azuremlschemasprod.azureedge.net/latest/MLTable.schema.json

Although I get the following information when selecting the folder with the above mentioned MLTable file:

Invalid MLTable format, convert_column_types does not support some of the provided properties: to_type. If you want to proceed, consider checking "Skip data validation" and click "Next." We will not validate your data path or query, or try to access your data for preview and schema checks.

Any ideas how to fix this?

The section of the schema I am relating to is this:

   ...  
                       {  
                           "type": "object",  
                           "properties": {  
                               "columns": {  
                                   "oneOf": [  
                                       {  
                                           "type": "string",  
                                           "description": "single column to be selected"  
                                       },  
                                       {  
                                           "type": "array",  
                                           "description": "list of columns to be selected",  
                                           "items": {  
                                               "type": "string"  
                                           }  
                                       },  
                                       {  
                                           "type": "object",  
                                           "description": "pattern and properties defining columns to be selected",  
                                           "properties": {  
                                               "pattern": {  
                                                   "type": "string"  
                                               },  
                                               "ignore_case": {  
                                                   "type": "boolean",  
                                                   "default": false  
                                               },  
                                               "invert": {  
                                                   "type": "boolean",  
                                                   "default": false  
                                               }  
                                           }  
                                       }  
                                   ]  
                               },  
                               "to_type": {  
                                   "type": "object",  
                                   "properties": {  
                                       "datetime": {  
                                           "type": "object",  
                                           "properties": {  
                                               "formats": {  
                                                   "type": "array",  
                                                   "items": {  
                                                       "type": "string"  
                                                   }  
                                               }  
                                           },  
                                           "additionalProperties": false  
                                       }  
                                   },  
                                   "additionalProperties": false  
                               }  
                           },  
                           "additionalProperties": false  
                       },  
   ...  
Not Monitored
Not Monitored
Tag not monitored by Microsoft.
39,141 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.