Quick Start#

ploomber-engine is a toolbox for executing notebooks.

It provides enhanced capabilities such as debugging, profiling and experiment tracking.

Installation#

pip install -U ploomber-engine

or

conda install ploomber-engine -c conda-forge

Run notebook#

Tip

There is a command-line interface available. Run: ploomber-engine --help to learn more.

Let’s download a sample notebook.

%%bash
curl https://raw.githubusercontent.com/ploomber/ploomber-engine/main/examples/display.ipynb --output nb.ipynb
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  2672  100  2672    0     0  14683      0 --:--:-- --:--:-- --:--:-- 14762
from ploomber_engine import execute_notebook

_ = execute_notebook("nb.ipynb", "output.ipynb", parameters=dict(x=1, y=2))
Hide code cell output
  0%|                                                    | 0/11 [00:00<?, ?it/s]
Executing cell: 1:   0%|                                 | 0/11 [00:00<?, ?it/s]
Executing cell: 2:   0%|                                 | 0/11 [00:00<?, ?it/s]
Executing cell: 2:  27%|██████▊                  | 3/11 [00:00<00:00, 14.71it/s]
Executing cell: 3:  27%|██████▊                  | 3/11 [00:00<00:00, 14.71it/s]
Executing cell: 4:  27%|██████▊                  | 3/11 [00:00<00:00, 14.71it/s]
sending something to standard error

Executing cell: 5:  27%|██████▊                  | 3/11 [00:00<00:00, 14.71it/s]
Executing cell: 5:  55%|█████████████▋           | 6/11 [00:00<00:00, 14.46it/s]
Executing cell: 6:  55%|█████████████▋           | 6/11 [00:00<00:00, 14.46it/s]
Executing cell: 7:  55%|█████████████▋           | 6/11 [00:00<00:00, 14.46it/s]
Executing cell: 7:  73%|██████████████████▏      | 8/11 [00:00<00:00, 11.52it/s]
Executing cell: 8:  73%|██████████████████▏      | 8/11 [00:00<00:00, 11.52it/s]
Executing cell: 9:  73%|██████████████████▏      | 8/11 [00:00<00:00, 11.52it/s]
Executing cell: 10:  73%|█████████████████▍      | 8/11 [00:00<00:00, 11.52it/s]
Executing cell: 10: 100%|███████████████████████| 11/11 [00:00<00:00, 16.68it/s]

Log print statements#

_ = execute_notebook("nb.ipynb", "output.ipynb", log_output=True, progress_bar=False)
sending something to standard error
some message
showing our logo :)

Plot cell’s runtime#

Note

Runtime profiling requires extra dependencies: pip install matplotlib

_ = execute_notebook(
    "nb.ipynb", "output.ipynb", profile_runtime=True, progress_bar=False
)
sending something to standard error
_images/f6c9a14caac712230f963aef2a9c596cedbc15c752dd23a83d864f921cb06dcf.png
ls output-*
output-runtime.png

Plot cell’s memory usage#

Note

Memory profiling requires extra dependencies: pip install matplotlib psutil

_ = execute_notebook(
    "nb.ipynb", "output.ipynb", profile_memory=True, progress_bar=False
)
sending something to standard error
_images/3210431c2f3ffbd0b412371a6331315e753098cec6fcaa790146a06a54fb47e1.png
ls output-*
output-memory-usage.png  output-runtime.png