Azure Databricks AutoML never reaches evaluate stage

andrew blance 1 Reputation point
2022-05-19T15:18:37.517+00:00

Hey all,

I am trying to use AutoML within Azure Databricks to perform a classification task. It successfully completes "configure", and runs for the entire length of the training time. It counts down 60 minutes, without throwing an error. However, at this point it sits at 00:00 permanently, never erroring or moving past this to the "evaluate" stage.

During the hour of training time, however, I am not sure any training happens. No training runs populate the table on the AutoML experiment page. It looks like nothing has happened during this time.

There is only one thing in the run table on the experiment page - a notebook called DataExploration. If I open this, I can run every cell and see a HTML output detailing the distribution of my data. Due to the size it has been stratified to 84.6%, but aside from that it all looks correct. Oddly, next to the notebook name in the table there is a small red cross - implying something has gone wrong. However, I cannot find any errors relating to this, and as I look within the notebook everything seems fine.

I am using a High Concurrency compute running the 10.4 LTS ML version of Databrick and PySpark. It is the Standard_F4, with 4 workers. I have read some places suggesting the high concurrency computes have difficulty importing some data when using AutoML, but the data can be imported and viewed in the DataExploration notebook, so I am not sure if the compute is my problem.

Can anyone please help me identify the problems - getting output and errors from AutoML seems a little obtuse, and what does come out is regularly vague. Any help would be very very much appreciated!

Thanks!

Not Monitored
Not Monitored
Tag not monitored by Microsoft.
42,053 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.