pandas boolean mask is ~8.13× slower than polars filter for filtering rows by a float threshold
Keeps roughly half of the rows.
Result
| Run | pandas boolean mask | polars filter | Ratio |
|---|---|---|---|
| Run 1 | 21.36 | 2.64 | 8.09× |
| Run 2 | 25.58 | 2.72 | 9.40× |
| Run 3 | 22.59 | 2.78 | 8.13× |
Spread over 3 runs: pandas boolean mask CV 9.4% / polars filter CV 2.6% / ratio CV 8.7%
The two calls
df[df['v'] > 0.5] df.filter(pl.col('v') > 0.5) Conditions
400,000 rows, float column, threshold 0.5.
Where this number does not apply
A highly selective filter (1% of rows) has a different cost because the copy dominates.
Reproduce
cd bench && python run.py --run 1 --out ../data/run1.json