Ctrl+K
Deploy AI apps for free on Ploomber Cloud!

API Reference

API Reference#

execute_notebook#

ploomber_engine.execute_notebook(input_path, output_path, *, parameters=None, log_output=False, profile_runtime=False, profile_memory=False, progress_bar=True, debug_later=False, verbose=False, remove_tagged_cells=None, cwd='.', save_profiling_data=False)#

Executes a notebook. Drop-in replacement for papermill.execute_notebook with enhanced capabilities.

Parameters:

  • input_path (str or Path or nbformat.NotebookNode) – Path to input notebook or NotebookNode object of notebook

  • output_path (str or Path or None) – Path to save executed notebook. If None, no file will be saved

  • parameters (dict, optional, default=None) – Arbitrary keyword arguments to pass to the notebook parameters

  • log_output (bool, optional, default=False) – Flag for whether or not to write notebook output to stdout

  • profile_runtime (bool or Path, default=False) – If True, profile cell’s runtime (stores plot in a .png file in the same folder as output_path). If Path, profiles and stores the plot to the given Path.

  • profile_memory (bool or Path, default=False) – If True, profile cell’s memory usage (stores plot in a .png file in the same folder as output_path). If Path, profiles and stores the plot to the given Path.

  • progress_bar (bool, default=True) – Display a progress bar.

  • debug_later (bool, default=False) – Serialize Python traceback for later debugging. The .dump file is stored next to the output notebook.

  • verbose (bool, default=False) – If True, prints information messages

  • remove_tagged_cells (str or list, default=None) – Cells with any of the passed tag(s) will be removed from the notebook before execution.

  • cwd (str or Path, default='.') – Working directory to use when executing the notebook

  • save_profiling_data (bool or Path, default=False) – If True, saves profiling data generated from profile_memory and profile_runtime (stores a .csv file in the same folder as output_path). If Path, saves profiling data to the given Path

Returns:

nb – Executed notebook object

Return type:

NotebookNode

Notes

Changed in version 0.0.31: Allow paths to be passed to profile_runtime, profile_memory arguments.

Changed in version 0.0.23: Added cwd argument. Added save_profiling_data argument.

Changed in version 0.0.21: Added remove_tagged_cells arguments.

Changed in version 0.0.19: Added parameters, progress_bar, debug_later, and verbose arguments.

New in version 0.0.18.

Examples

Execute a notebook:

>>> from ploomber_engine import execute_notebook
>>> out = execute_notebook("nb.ipynb", "out.ipynb")

Display any printed messages:

>>> from ploomber_engine import execute_notebook
>>> out = execute_notebook("nb.ipynb", "out.ipynb", log_output=True)

Store a plot with cell’s runtime:

>>> from ploomber_engine import execute_notebook
>>> out = execute_notebook("nb.ipynb", "out.ipynb", profile_runtime=True)

Store a plot with cell’s memory usage:

>>> from ploomber_engine import execute_notebook
>>> out = execute_notebook("nb.ipynb", "out.ipynb", profile_memory=True)

Store a plot with cell’s memory usage at a custom path:

>>> from ploomber_engine import execute_notebook
>>> out = execute_notebook("nb.ipynb", "out.ipynb", profile_memory='memory.png')

Remove cells with the tag “remove” before execution:

>>> from ploomber_engine import execute_notebook
>>> out = execute_notebook("nb.ipynb", "out.ipynb", remove_tagged_cells="remove")

Remove cells with any of the passed tags before execution:

>>> from ploomber_engine import execute_notebook
>>> out = execute_notebook("nb.ipynb", "out.ipynb",
...                        remove_tagged_cells=["remove", "also-remove"])

PloomberClient#

class ploomber_engine.ipython.PloomberClient(nb, display_stdout=False, progress_bar=True, debug_later=False, remove_tagged_cells=None, cwd='.')#

PloomberClient executes Jupyter notebooks

Parameters:

  • nb – Notebook object

  • display_output (bool, default=False) – If True, it prints the same output the notebook prints.

  • progress_bar (bool, default=True) – Display a progress bar.

  • debug_later (bool or str, default=False) – Serialize Python traceback for later debugging. If True, it stores the traceback at jupyter.dump, if a string, it stores the traceback there.

  • remove_tagged_cells (str or list, default=None) – Cells with any of the passed tag(s) will be removed from the notebook before execution.

  • cwd (str or Path, default='.') – Working directory to use when executing the notebook

Notes

Changed in version 0.0.25dev: Removed cell outputs and execution count Added error messages to output notebook during an exception Temporary Fixed the stderr and stdout streams of ploomber engine

Changed in version 0.0.23: Added cwd argument.

Changed in version 0.0.21: Added remove_tagged_cells arguments.

Changed in version 0.0.19: Added progress_bar and debug_later arguments.

Examples

Execute notebook:

>>> from ploomber_engine.ipython import PloomberClient
>>> import nbformat
>>> nb = nbformat.v4.new_notebook()
>>> cell = nbformat.v4.new_code_cell("1+1")
>>> nb.cells = [cell]
>>> client = PloomberClient(nb)
>>> out = client.execute()
>>> out.cells[0]['outputs'][0]['data']
{'text/plain': '2'}

Remove cells tagged “remove” before execution:

>>> from ploomber_engine.ipython import PloomberClient
>>> import nbformat
>>> nb = nbformat.v4.new_notebook()
>>> nb.cells = [nbformat.v4.new_code_cell("1+1"),
...             nbformat.v4.new_code_cell("1/0",
...             metadata=dict(tags=["remove"]))]
>>> client = PloomberClient(nb, remove_tagged_cells="remove")
>>> out = client.execute()
>>> out.cells[0]['outputs'][0]['data']
{'text/plain': '2'}

Remove cells tagged with any of the passed tags before execution:

>>> from ploomber_engine.ipython import PloomberClient
>>> import nbformat
>>> nb = nbformat.v4.new_notebook()
>>> nb.cells = [nbformat.v4.new_code_cell("1+1"),
...             nbformat.v4.new_code_cell("1/0",
...             metadata=dict(tags=["remove"])),
...             nbformat.v4.new_code_cell("2/0",
...             metadata=dict(tags=["also-remove"]))
...             ]
>>> client = PloomberClient(nb, remove_tagged_cells=["remove", "also-remove"])
>>> out = client.execute()
>>> out.cells[0]['outputs'][0]['data']
{'text/plain': '2'}
execute(parameters=None)#

Execute the notebook

Returns:

Notebook object

Return type:

nb

Notes

Changed in version 0.0.25dev: Added error messages to output notebook during an exception

Changed in version 0.0.19: Added parameters argument

classmethod from_path(path, display_stdout=False, progress_bar=True, debug_later=False, remove_tagged_cells=None, cwd='.')#

Initialize client from a path to a notebook

Parameters:

  • path (str) – Path to the .ipynb file

  • display_output (bool, default=False) – If True, it prints the same output the notebook prints.

  • progress_bar (bool, default=True) – Display a progress bar.

  • debug_later (bool or str, default=False) – Serialize Python traceback for later debugging. If True, it stores the traceback at jupyter.dump, if a string, it stores the traceback there.

  • remove_tagged_cells (str or list, default=None) – Cells with any of the passed tag(s) will be removed from the notebook before execution.

  • cwd (str or Path, default='.') – Working directory to use when executing the notebook

Notes

Changed in version 0.0.23: Added cwd argument.

Changed in version 0.0.21: Added remove_tagged_cells arguments.

Changed in version 0.0.19: Added progress_bar and debug_later arguments.

Changed in version 0.0.18: Added display_stdout argument.

Examples

>>> from ploomber_engine.ipython import PloomberClient
>>> client = PloomberClient.from_path("nb.ipynb")
>>> out = client.execute()
get_definitions()#

Returns class and function definitions without running the notebook

Examples

>>> from ploomber_engine.ipython import PloomberClient
>>> import nbformat
>>> nb = nbformat.v4.new_notebook()
>>> first = nbformat.v4.new_code_cell("def add(x, y): return x + y")
>>> second = nbformat.v4.new_code_cell("def multiply(x, y): return x * y")
>>> nb.cells = [first, second]
>>> client = PloomberClient(nb)
>>> defs = client.get_definitions()
>>> add = defs["add"]
>>> add(1, 41)
42
>>> multiply = defs["multiply"]
>>> multiply(2, 21)
42
get_namespace(namespace=None)#

Run the notebook and return all the output variables

Parameters:

namespace (dict, default=None) – The initial namespace. It can be used to set initial values prior to execution.

Examples

Execute notebook and get the output variables:

>>> from ploomber_engine.ipython import PloomberClient
>>> import nbformat
>>> nb = nbformat.v4.new_notebook()
>>> first = nbformat.v4.new_code_cell("x = 1")
>>> second = nbformat.v4.new_code_cell("y = 41")
>>> third = nbformat.v4.new_code_cell("z = x + y")
>>> nb.cells = [first, second, third]
>>> client = PloomberClient(nb)
>>> ns = client.get_namespace()
>>> ns["x"]
1
>>> ns["y"]
41
>>> ns["z"]
42

Notes

Changed in version 0.0.22: Added namespace arguments.

ploomber_engine.profiling#

ploomber_engine.profiling.plot_memory_usage(nb)#

Plot cell memory usage. Notebook must contain “memory_usage” under the “ploomber” key in the metadata

Notes

New in version 0.0.18.

ploomber_engine.profiling.plot_cell_runtime(nb)#

Plot cell runtime

Notes

New in version 0.0.18.

ploomber_engine.testing#

ploomber_engine.testing.test_notebook(path_to_nb)#

Test a notebook by running it and comparing produced with existing outputs

Notes

New in version 0.0.16.

Contents