API Reference#
execute_notebook
#
- ploomber_engine.execute_notebook(input_path, output_path, *, parameters=None, log_output=False, profile_runtime=False, profile_memory=False, progress_bar=True, debug_later=False, verbose=False, remove_tagged_cells=None, cwd='.', save_profiling_data=False)#
Executes a notebook. Drop-in replacement for
papermill.execute_notebook
with enhanced capabilities.- Parameters:
input_path (str or Path or nbformat.NotebookNode) – Path to input notebook or NotebookNode object of notebook
output_path (str or Path or None) – Path to save executed notebook. If None, no file will be saved
parameters (dict, optional, default=None) – Arbitrary keyword arguments to pass to the notebook parameters
log_output (bool, optional, default=False) – Flag for whether or not to write notebook output to stdout
profile_runtime (bool, optional, default=False) – If True, profile cell’s runtime (stores plot in a
.png
file in the same folder asoutput_path
)profile_memory (bool, optional, default=False) – If True, profile cell’s memory usage (stores plot in a
.png
file in the same folder asoutput_path
)progress_bar (bool, default=True) – Display a progress bar.
debug_later (bool, default=False) – Serialize Python traceback for later debugging. The
.dump
file is stored next to the output notebook.verbose (bool, default=False) – If True, prints information messages
remove_tagged_cells (str or list, default=None) – Cells with any of the passed tag(s) will be removed from the notebook before execution.
cwd (str or Path, default='.') – Working directory to use when executing the notebook
save_profiling_data (bool, default=False) – If True, saves profiling data generated from profile_memory and profile_runtime (stores a
.csv
file in the same folder asoutput_path
)
- Returns:
nb – Executed notebook object
- Return type:
NotebookNode
Notes
Changed in version 0.0.23: Added
cwd
argument. Addedsave_profiling_data
argument.Changed in version 0.0.21: Added
remove_tagged_cells
arguments.Changed in version 0.0.19: Added
parameters
,progress_bar
,debug_later
, andverbose
arguments.New in version 0.0.18.
Examples
Execute a notebook:
>>> from ploomber_engine import execute_notebook >>> out = execute_notebook("nb.ipynb", "out.ipynb")
Display any printed messages:
>>> from ploomber_engine import execute_notebook >>> out = execute_notebook("nb.ipynb", "out.ipynb", log_output=True)
Store a plot with cell’s runtime:
>>> from ploomber_engine import execute_notebook >>> out = execute_notebook("nb.ipynb", "out.ipynb", profile_runtime=True)
Store a plot with cell’s memory usage:
>>> from ploomber_engine import execute_notebook >>> out = execute_notebook("nb.ipynb", "out.ipynb", profile_memory=True)
Remove cells with the tag “remove” before execution:
>>> from ploomber_engine import execute_notebook >>> out = execute_notebook("nb.ipynb", "out.ipynb", remove_tagged_cells="remove")
Remove cells with any of the passed tags before execution:
>>> from ploomber_engine import execute_notebook >>> out = execute_notebook("nb.ipynb", "out.ipynb", ... remove_tagged_cells=["remove", "also-remove"])
PloomberClient
#
- class ploomber_engine.ipython.PloomberClient(nb, display_stdout=False, progress_bar=True, debug_later=False, remove_tagged_cells=None, cwd='.')#
PloomberClient executes Jupyter notebooks
- Parameters:
nb – Notebook object
display_output (bool, default=False) – If True, it prints the same output the notebook prints.
progress_bar (bool, default=True) – Display a progress bar.
debug_later (bool or str, default=False) – Serialize Python traceback for later debugging. If True, it stores the traceback at
jupyter.dump
, if a string, it stores the traceback there.remove_tagged_cells (str or list, default=None) – Cells with any of the passed tag(s) will be removed from the notebook before execution.
cwd (str or Path, default='.') – Working directory to use when executing the notebook
Notes
Changed in version 0.0.25dev: Removed cell outputs and execution count Added error messages to output notebook during an exception Temporary Fixed the stderr and stdout streams of ploomber engine
Changed in version 0.0.23: Added
cwd
argument.Changed in version 0.0.21: Added
remove_tagged_cells
arguments.Changed in version 0.0.19: Added
progress_bar
anddebug_later
arguments.Examples
Execute notebook:
>>> from ploomber_engine.ipython import PloomberClient >>> import nbformat >>> nb = nbformat.v4.new_notebook() >>> cell = nbformat.v4.new_code_cell("1+1") >>> nb.cells = [cell] >>> client = PloomberClient(nb) >>> out = client.execute() >>> out.cells[0]['outputs'][0]['data'] {'text/plain': '2'}
Remove cells tagged “remove” before execution:
>>> from ploomber_engine.ipython import PloomberClient >>> import nbformat >>> nb = nbformat.v4.new_notebook() >>> nb.cells = [nbformat.v4.new_code_cell("1+1"), ... nbformat.v4.new_code_cell("1/0", ... metadata=dict(tags=["remove"]))] >>> client = PloomberClient(nb, remove_tagged_cells="remove") >>> out = client.execute() >>> out.cells[0]['outputs'][0]['data'] {'text/plain': '2'}
Remove cells tagged with any of the passed tags before execution:
>>> from ploomber_engine.ipython import PloomberClient >>> import nbformat >>> nb = nbformat.v4.new_notebook() >>> nb.cells = [nbformat.v4.new_code_cell("1+1"), ... nbformat.v4.new_code_cell("1/0", ... metadata=dict(tags=["remove"])), ... nbformat.v4.new_code_cell("2/0", ... metadata=dict(tags=["also-remove"])) ... ] >>> client = PloomberClient(nb, remove_tagged_cells=["remove", "also-remove"]) >>> out = client.execute() >>> out.cells[0]['outputs'][0]['data'] {'text/plain': '2'}
- execute(parameters=None)#
Execute the notebook
- Returns:
Notebook object
- Return type:
nb
Notes
Changed in version 0.0.25dev: Added error messages to output notebook during an exception
Changed in version 0.0.19: Added
parameters
argument
- classmethod from_path(path, display_stdout=False, progress_bar=True, debug_later=False, remove_tagged_cells=None, cwd='.')#
Initialize client from a path to a notebook
- Parameters:
path (str) – Path to the
.ipynb
filedisplay_output (bool, default=False) – If True, it prints the same output the notebook prints.
progress_bar (bool, default=True) – Display a progress bar.
debug_later (bool or str, default=False) – Serialize Python traceback for later debugging. If True, it stores the traceback at
jupyter.dump
, if a string, it stores the traceback there.remove_tagged_cells (str or list, default=None) – Cells with any of the passed tag(s) will be removed from the notebook before execution.
cwd (str or Path, default='.') – Working directory to use when executing the notebook
Notes
Changed in version 0.0.23: Added
cwd
argument.Changed in version 0.0.21: Added
remove_tagged_cells
arguments.Changed in version 0.0.19: Added
progress_bar
anddebug_later
arguments.Changed in version 0.0.18: Added
display_stdout
argument.Examples
>>> from ploomber_engine.ipython import PloomberClient >>> client = PloomberClient.from_path("nb.ipynb") >>> out = client.execute()
- get_definitions()#
Returns class and function definitions without running the notebook
Examples
>>> from ploomber_engine.ipython import PloomberClient >>> import nbformat >>> nb = nbformat.v4.new_notebook() >>> first = nbformat.v4.new_code_cell("def add(x, y): return x + y") >>> second = nbformat.v4.new_code_cell("def multiply(x, y): return x * y") >>> nb.cells = [first, second] >>> client = PloomberClient(nb) >>> defs = client.get_definitions() >>> add = defs["add"] >>> add(1, 41) 42 >>> multiply = defs["multiply"] >>> multiply(2, 21) 42
- get_namespace(namespace=None)#
Run the notebook and return all the output variables
- Parameters:
namespace (dict, default=None) – The initial namespace. It can be used to set initial values prior to execution.
Examples
Execute notebook and get the output variables:
>>> from ploomber_engine.ipython import PloomberClient >>> import nbformat >>> nb = nbformat.v4.new_notebook() >>> first = nbformat.v4.new_code_cell("x = 1") >>> second = nbformat.v4.new_code_cell("y = 41") >>> third = nbformat.v4.new_code_cell("z = x + y") >>> nb.cells = [first, second, third] >>> client = PloomberClient(nb) >>> ns = client.get_namespace() >>> ns["x"] 1 >>> ns["y"] 41 >>> ns["z"] 42
Notes
Changed in version 0.0.22: Added
namespace
arguments.
ploomber_engine.profiling
#
- ploomber_engine.profiling.plot_memory_usage(nb)#
Plot cell memory usage. Notebook must contain “memory_usage” under the “ploomber” key in the metadata
Notes
New in version 0.0.18.
- ploomber_engine.profiling.plot_cell_runtime(nb)#
Plot cell runtime
Notes
New in version 0.0.18.
ploomber_engine.testing
#
- ploomber_engine.testing.test_notebook(path_to_nb)#
Test a notebook by running it and comparing produced with existing outputs
Notes
New in version 0.0.16.