getting access to std_log.txt programmatically

Question

getting access to std_log.txt programmatically

ironv 0

I am running my experiments on a compute cluster using azure.ai.ml.command. My script uses lightgbm and Optuna for hyperparameter tuning. Intermediate output is written to stdout (not using logging package) which I am able to see in std_log.txt in the UI. Is there a way to programmatically grab this file and move it to a different location?

Screenshot 2023-05-03 at 1.03.54 PM

Ramr-msft 17,826 Reputation points

2023-05-04T11:45:14.93+00:00

@ironv Thanks for the question, For the pipelines the outputs of each step can be accessed via the .outputs property or use the get_output() method on the run. You can use the Download all option from the UI or Download of job.

ironv 0

What you are suggesting does NOT work for scipts run using command.

status = list()
for sz in range(1,3):
  cmd = "python hello_pandas.py --read_dir ${{inputs.input}} --write_dir ${{outputs.output}}" + f" --size {sz*10000}"
  job = command(code=".", command=cmd, experiment_name="cpu-test", environment="py310cpu:1",
                  compute="d12v2-c0-d", inputs=dict(input=input), outputs=dict(output=output))
  status.append(ml_client.create_or_update(job))

status[0] does not have a get_output(). It does have .outputs but that has reference to an object.

Ramr-msft 17,826 Reputation points

2023-05-08T01:02:49.3933333+00:00

@ironv Thanks for the details. Can you please add more details about the location that you want to move. We are checking internally for scripts run using the command.
ironv 0 Reputation points

2023-05-08T03:49:23.38+00:00

I'd like to move it to /mnt/azureblobshare/users/firstname.lastname. But that isn't the point - is there a way for me to do this programmatically say to the same location as --write_dir? Thx!
ironv 0 Reputation points

2023-05-09T00:38:22.1733333+00:00

Thx! Can you please point me to an example which uses this?
Ramr-msft 17,826 Reputation points

2023-05-29T01:31:52.57+00:00

@ironv Thanks, We have forwarded to the team to add examples for this scenario.
Drizen, Jordan 0 Reputation points

2024-05-03T20:12:38.5466667+00:00

Was this ever resolved? I am able to grab this file programattically thru ADLS, but only after the step has completed. I want to get live logs.

1 answer

Your answer

Ramr-msft 17,826 Reputation points

2023-05-04T11:45:14.93+00:00

@ironv Thanks for the question, For the pipelines the outputs of each step can be accessed via the .outputs property or use the get_output() method on the run. You can use the Download all option from the UI or Download of job.
ironv 0 Reputation points

2023-05-05T22:59:22.41+00:00

What you are suggesting does NOT work for scipts run using command.

status = list() for sz in range(1,3): cmd = "python hello_pandas.py --read_dir ${{inputs.input}} --write_dir ${{outputs.output}}" + f" --size {sz*10000}" job = command(code=".", command=cmd, experiment_name="cpu-test", environment="py310cpu:1", compute="d12v2-c0-d", inputs=dict(input=input), outputs=dict(output=output)) status.append(ml_client.create_or_update(job))

status[0] does not have a get_output(). It does have .outputs but that has reference to an object.
Ramr-msft 17,826 Reputation points

2023-05-08T01:02:49.3933333+00:00

@ironv Thanks for the details. Can you please add more details about the location that you want to move. We are checking internally for scripts run using the command.
ironv 0 Reputation points

2023-05-08T03:49:23.38+00:00

I'd like to move it to /mnt/azureblobshare/users/firstname.lastname. But that isn't the point - is there a way for me to do this programmatically say to the same location as --write_dir? Thx!
ironv 0 Reputation points

2023-05-09T00:38:22.1733333+00:00

Thx! Can you please point me to an example which uses this?
Ramr-msft 17,826 Reputation points

2023-05-29T01:31:52.57+00:00

@ironv Thanks, We have forwarded to the team to add examples for this scenario.
Drizen, Jordan 0 Reputation points

2024-05-03T20:12:38.5466667+00:00

Was this ever resolved? I am able to grab this file programattically thru ADLS, but only after the step has completed. I want to get live logs.

Answer 1

Ramr-msft 17,826

@ironv Thanks for the details, You can use the Job Download method for the output logs. Here is the link: https://learn.microsoft.com/en-us/python/api/azure-ai-ml/azure.ai.ml.operations.joboperations?view=azure-python#azure-ai-ml-operations-joboperations-download

Anonymous

2023-09-15T06:28:16.1433333+00:00
Hello @Ramr-msft I was this using functionality from last few months but recently started facing some issue with the below code. I'm unable to download the predictions result. However, I can see the prediction output saved in Azure Blob Storage. Could you pls help me out with this?

ml_client.jobs.download(batch_job.name, download_path= f"{batch_prediction_dir}/csv/", output_name="predictions")

Share via

getting access to std_log.txt programmatically

1 answer

Your answer