API Reference

execute_notebook

ploomber_engine.execute_notebook(input_path, output_path, *, parameters=None, log_output=False, profile_runtime=False, profile_memory=False, progress_bar=True, debug_later=False, verbose=False, remove_tagged_cells=None, cwd='.', save_profiling_data=False)

Executes a notebook. Drop-in replacement for papermill.execute_notebook with enhanced capabilities.

Parameters:

• input_path (str or Path or nbformat.NotebookNode) – Path to input notebook or NotebookNode object of notebook

• output_path (str or Path or None) – Path to save executed notebook. If None, no file will be saved

• parameters (dict, optional, default=None) – Arbitrary keyword arguments to pass to the notebook parameters

• log_output (bool, optional, default=False) – Flag for whether or not to write notebook output to stdout

• profile_runtime (bool, optional, default=False) – If True, profile cell’s runtime (stores plot in a .png file in the same folder as output_path)

• profile_memory (bool, optional, default=False) – If True, profile cell’s memory usage (stores plot in a .png file in the same folder as output_path)

• progress_bar (bool, default=True) – Display a progress bar.

• debug_later (bool, default=False) – Serialize Python traceback for later debugging. The .dump file is stored next to the output notebook.

• verbose (bool, default=False) – If True, prints information messages

• remove_tagged_cells (str or list, default=None) – Cells with any of the passed tag(s) will be removed from the notebook before execution.

• cwd (str or Path, default='.') – Working directory to use when executing the notebook

• save_profiling_data (bool, default=False) – If True, saves profiling data generated from profile_memory and profile_runtime (stores a .csv file in the same folder as output_path)

Returns:

nb – Executed notebook object

Return type:

NotebookNode

Notes

Changed in version 0.0.23: Added cwd argument. Added save_profiling_data argument.

Changed in version 0.0.21: Added remove_tagged_cells arguments.

Changed in version 0.0.19: Added parameters, progress_bar, debug_later, and verbose arguments.

New in version 0.0.18.

Examples

Execute a notebook:

>>> from ploomber_engine import execute_notebook
>>> out = execute_notebook("nb.ipynb", "out.ipynb")

Display any printed messages:

>>> from ploomber_engine import execute_notebook
>>> out = execute_notebook("nb.ipynb", "out.ipynb", log_output=True)

Store a plot with cell’s runtime:

>>> from ploomber_engine import execute_notebook
>>> out = execute_notebook("nb.ipynb", "out.ipynb", profile_runtime=True)

Store a plot with cell’s memory usage:

>>> from ploomber_engine import execute_notebook
>>> out = execute_notebook("nb.ipynb", "out.ipynb", profile_memory=True)

Remove cells with the tag “remove” before execution:

>>> from ploomber_engine import execute_notebook
>>> out = execute_notebook("nb.ipynb", "out.ipynb", remove_tagged_cells="remove")

Remove cells with any of the passed tags before execution:

>>> from ploomber_engine import execute_notebook
>>> out = execute_notebook("nb.ipynb", "out.ipynb",
...                        remove_tagged_cells=["remove", "also-remove"])

PloomberClient

class ploomber_engine.ipython.PloomberClient(nb, display_stdout=False, progress_bar=True, debug_later=False, remove_tagged_cells=None, cwd='.')

PloomberClient executes Jupyter notebooks

Parameters:

• nb – Notebook object

• display_output (bool, default=False) – If True, it prints the same output the notebook prints.

• progress_bar (bool, default=True) – Display a progress bar.

• debug_later (bool or str, default=False) – Serialize Python traceback for later debugging. If True, it stores the traceback at jupyter.dump, if a string, it stores the traceback there.

• remove_tagged_cells (str or list, default=None) – Cells with any of the passed tag(s) will be removed from the notebook before execution.

• cwd (str or Path, default='.') – Working directory to use when executing the notebook

Notes

Changed in version 0.0.25dev: Removed cell outputs and execution count Added error messages to output notebook during an exception Temporary Fixed the stderr and stdout streams of ploomber engine

Changed in version 0.0.23: Added cwd argument.

Changed in version 0.0.21: Added remove_tagged_cells arguments.

Changed in version 0.0.19: Added progress_bar and debug_later arguments.

Examples

Execute notebook:

>>> from ploomber_engine.ipython import PloomberClient
>>> import nbformat
>>> nb = nbformat.v4.new_notebook()
>>> cell = nbformat.v4.new_code_cell("1+1")
>>> nb.cells = [cell]
>>> client = PloomberClient(nb)
>>> out = client.execute()
>>> out.cells[0]['outputs'][0]['data']
{'text/plain': '2'}

Remove cells tagged “remove” before execution:

>>> from ploomber_engine.ipython import PloomberClient
>>> import nbformat
>>> nb = nbformat.v4.new_notebook()
>>> nb.cells = [nbformat.v4.new_code_cell("1+1"),
...             nbformat.v4.new_code_cell("1/0",
...             metadata=dict(tags=["remove"]))]
>>> client = PloomberClient(nb, remove_tagged_cells="remove")
>>> out = client.execute()
>>> out.cells[0]['outputs'][0]['data']
{'text/plain': '2'}

Remove cells tagged with any of the passed tags before execution:

>>> from ploomber_engine.ipython import PloomberClient
>>> import nbformat
>>> nb = nbformat.v4.new_notebook()
>>> nb.cells = [nbformat.v4.new_code_cell("1+1"),
...             nbformat.v4.new_code_cell("1/0",
...             metadata=dict(tags=["remove"])),
...             nbformat.v4.new_code_cell("2/0",
...             metadata=dict(tags=["also-remove"]))
...             ]
>>> client = PloomberClient(nb, remove_tagged_cells=["remove", "also-remove"])
>>> out = client.execute()
>>> out.cells[0]['outputs'][0]['data']
{'text/plain': '2'}

execute(parameters=None)

Execute the notebook

Returns:

Notebook object

Return type:

nb

Notes

Changed in version 0.0.25dev: Added error messages to output notebook during an exception

Changed in version 0.0.19: Added parameters argument

classmethod from_path(path, display_stdout=False, progress_bar=True, debug_later=False, remove_tagged_cells=None, cwd='.')

Initialize client from a path to a notebook

Parameters:

• path (str) – Path to the .ipynb file

• display_output (bool, default=False) – If True, it prints the same output the notebook prints.

• progress_bar (bool, default=True) – Display a progress bar.

• debug_later (bool or str, default=False) – Serialize Python traceback for later debugging. If True, it stores the traceback at jupyter.dump, if a string, it stores the traceback there.

• remove_tagged_cells (str or list, default=None) – Cells with any of the passed tag(s) will be removed from the notebook before execution.

• cwd (str or Path, default='.') – Working directory to use when executing the notebook

Notes

Changed in version 0.0.23: Added cwd argument.

Changed in version 0.0.21: Added remove_tagged_cells arguments.

Changed in version 0.0.19: Added progress_bar and debug_later arguments.

Changed in version 0.0.18: Added display_stdout argument.

Examples

>>> from ploomber_engine.ipython import PloomberClient
>>> client = PloomberClient.from_path("nb.ipynb")
>>> out = client.execute()

get_definitions()

Returns class and function definitions without running the notebook

Examples

>>> from ploomber_engine.ipython import PloomberClient
>>> import nbformat
>>> nb = nbformat.v4.new_notebook()
>>> first = nbformat.v4.new_code_cell("def add(x, y): return x + y")
>>> second = nbformat.v4.new_code_cell("def multiply(x, y): return x * y")
>>> nb.cells = [first, second]
>>> client = PloomberClient(nb)
>>> defs = client.get_definitions()
>>> add = defs["add"]
>>> add(1, 41)
42
>>> multiply = defs["multiply"]
>>> multiply(2, 21)
42

get_namespace(namespace=None)

Run the notebook and return all the output variables

Parameters:

namespace (dict, default=None) – The initial namespace. It can be used to set initial values prior to execution.

Examples

Execute notebook and get the output variables:

>>> from ploomber_engine.ipython import PloomberClient
>>> import nbformat
>>> nb = nbformat.v4.new_notebook()
>>> first = nbformat.v4.new_code_cell("x = 1")
>>> second = nbformat.v4.new_code_cell("y = 41")
>>> third = nbformat.v4.new_code_cell("z = x + y")
>>> nb.cells = [first, second, third]
>>> client = PloomberClient(nb)
>>> ns = client.get_namespace()
>>> ns["x"]
1
>>> ns["y"]
41
>>> ns["z"]
42

Notes

Changed in version 0.0.22: Added namespace arguments.

ploomber_engine.profiling

ploomber_engine.profiling.plot_memory_usage(nb)

Plot cell memory usage. Notebook must contain “memory_usage” under the “ploomber” key in the metadata

Notes

New in version 0.0.18.

ploomber_engine.profiling.plot_cell_runtime(nb)

Plot cell runtime

Notes

New in version 0.0.18.

ploomber_engine.testing

ploomber_engine.testing.test_notebook(path_to_nb)

Test a notebook by running it and comparing produced with existing outputs

Notes

New in version 0.0.16.