pandas groupby + sum is ~3.45× slower than polars group_by + agg for summing a column per group
Operates on 1,000 groups over 400,000 rows.
Result
| Run | pandas groupby + sum | polars group_by + agg | Ratio |
|---|---|---|---|
| Run 1 | 109.68 | 32.08 | 3.42× |
| Run 2 | 114.42 | 30.41 | 3.76× |
| Run 3 | 105.64 | 30.59 | 3.45× |
Spread over 3 runs: pandas groupby + sum CV 4.0% / polars group_by + agg CV 3.0% / ratio CV 5.3%
The two calls
df.groupby('g', as_index=False)['v'].sum()df.group_by('g').agg(pl.col('v').sum())Conditions
Same frame, integer group key, float value column.
Where this number does not apply
Neither side is allowed to use a sorted key, so both pay for hashing.
Reproduce
cd bench && python run.py --run 1 --out ../data/run1.json