how can i pass the output of a notebooks runner step as an input? (the executed notebook including the cells outputs)

David Nuñez 0 Puntos de reputación
2024-04-24T23:55:35.23+00:00

I'm having problems with the flow of inputs/outputs dependencies in a pipeline.

Im trying to run a ipynb file in a step (NotebookRunnerStep) and later use that notebook

(the ipynb file, including all the cell outputs generated when running) as an input of the next step, but i keep getting errors.

I would greatly appreciate a 10-minute video call session to resolve doubts.

output_from_notebook = PipelineData(name="notebook_output",
                                    datastore= def_blob_store,
                                    output_path_on_compute= 'notebook_output')

step1 = NotebookRunnerStep(name="processing_step",
                                          notebook_run_config=cfg,
                                          params={"param_name": param_variable},
                                          allow_reuse=False,
                                          compute_target=aml_compute,
                                          output_notebook_pipeline_data_name='notebook_output',
                                          outputs = [output_from_notebook])

html_output = PipelineData(name="html_output", datastore=def_blob_store, output_path_on_compute='html_output', is_directory=False)

step2 = PythonScriptStep(name="convert_step",
                         arguments=[
                                '--notebook', output_from_notebook.as_mount(), 
                                '--output_html', source_directory 
                            ],
                         inputs=[output_from_notebook],
                         outputs = [html_output],
                         script_name='convert_notebook_to_html.py', 
                         compute_target=aml_compute, 
                         source_directory=source_directory,
                         allow_reuse=False,
                         runconfig = run_config_cd)
Azure
Azure
Plataforma e infraestructura de informática en la nube para crear, implementar y administrar aplicaciones y servicios a través de una red mundial de centros de datos administrados por Microsoft.
370 preguntas
0 comentarios No hay comentarios
{count} votos

Su respuesta

Las respuestas se pueden marcar como respuestas aceptadas por el autor de la pregunta, lo que ayuda a los usuarios a conocer la respuesta que resolvió el problema del autor.