pd.concat is ~16.94× slower than pl.concat for stacking a frame onto itself
Two 400,000-row frames of two columns each.
Result
| Run | pd.concat | pl.concat | Ratio |
|---|---|---|---|
| Run 1 | 4.09 | 0.26 | 15.70× |
| Run 2 | 4.13 | 0.24 | 16.94× |
| Run 3 | 5.61 | 0.33 | 17.20× |
Spread over 3 runs: pd.concat CV 18.8% / pl.concat CV 15.6% / ratio CV 4.8%
The two calls
pd.concat([a, b], ignore_index=True)pl.concat([a, b])Conditions
The inputs are already in memory; this is not an I/O test.
Where this number does not apply
Concatenating along columns, or with schema mismatch, is a different path.
Reproduce
cd bench && python run.py --run 1 --out ../data/run1.json