Protocol

Every number on this site comes from the same script, run three separate times.

Method

1. One sample is one operation on data that is already built; data construction is never inside the timer.
2. Seven samples per run, A and B alternating inside the run, so thermal or allocator drift hits both sides.
3. The sample value is the median, never the mean.
4. Three independent processes; the ratio's coefficient of variation across them must be ≤10% or the case is not published.
5. When the faster side of a case finishes under 20 ms, the operation is repeated inside the same sample (same count on both sides, capped at 60×) and the reported milliseconds are divided back to a single pass. This raises the timer resolution without changing what is being compared.
6. Cases that still fail the 10% rule are listed on /cases/ with the reason, never deleted.

What we do not measure

• Memory use (no heap instrumentation).
• Cluster or distributed execution.
• Other row counts: the ratio is not guaranteed to hold at 1,000 or 10,000,000 rows.
• Disk speed: the CSV cases touch a warm page cache on both sides.

Environment

Python 3.12.14 · pandas 3.0.1 · polars 2.1.0 · Windows 11 · single process, no threads pinned.

How to reproduce

cd bench
python run.py --run 1 --out ../data/run1.json
python run.py --run 2 --out ../data/run2.json
python run.py --run 3 --out ../data/run3.json
python aggregate.py