Azure Machine Learning: I cannot find experiment's user logs located in logs/user folder

2022-02-23T13:05:39.517+00:00

I am running experiments in Azure Machine Learning using ParallelRunStep, and I cannot get the user folder with logs as defined in readme.txt file with the log folder structure.
I cannot find log/user folder with "Logs generated when loading and running user's scripts."

readme.txt file states:
ParallelRunStep has two major parts:

  1. Scheduling, progress tracking and file concatenation for append_row.
  2. Processing mini batch by calling the entry script.
    The agent manager on each node start agents.
    An agent gets mini batch and calls the entry script against the mini batch. The "logs" folder has user, sys and perf sub folders. The user folder includes messages from the entry script in processing mini batches. The sys folder includes messages from #1 and non-entry script log from #2. The perf folder includes periodical checking result of resource usage.

In majority case, users can find the processing messages from the user folder.
Users need to check sys folder for messages beyond processing mini batches.
logs/
azureml/: Logs from azureml dependencies. e.g. azureml.dataprep
user/ : Logs generated when loading and running user's scripts.
error/ : Logs of errors encountered while loading and running entry script.
stderr/ : stderr output of user's scripts.
stdout/ : stdout output of user's scripts.
entry_script_log/ : Logs generated by loggers of EntryScript()
<node seq> :
processNNN.log.txt : Logs generated by loggers of EntryScript() from each process.

Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,921 questions
{count} vote

Accepted answer
  1. Ramr-msft 17,736 Reputation points
    2022-02-24T02:37:03.417+00:00

    @Calabria Montero, Salvador (SGRE SE D FP&DC WEF) Thanks for the question. Please follow the doc to view and log files for a run. Interactive logging sessions are typically used in notebook environments. The method Experiment.start_logging() starts an interactive logging session. Any metrics logged during the session are added to the run record in the experiment. The method run.complete() ends the sessions and marks the run as completed.

    https://learn.microsoft.com/en-us/azure/machine-learning/how-to-log-view-metrics#view-and-download-log-files-for-a-run


0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.