pandas str.rsplit + index is ~15.70× slower than polars str.split + list.get for splitting a string column
Splits on a hyphen and keeps the first piece.
Result
| Run | pandas str.rsplit + index | polars str.split + list.get | Ratio |
|---|---|---|---|
| Run 1 | 459.83 | 29.29 | 15.70× |
| Run 2 | 458.4 | 30.29 | 15.14× |
| Run 3 | 564.18 | 34.35 | 16.42× |
Spread over 3 runs: pandas str.rsplit + index CV 12.3% / polars str.split + list.get CV 8.6% / ratio CV 4.1%
The two calls
df['s'].str.rsplit('-', n=1).str[0] df['s'].str.split('-').list.get(0) Conditions
400,000 strings with two hyphens each.
Where this number does not apply
Splitting into a list column and exploding it is a different operation.
Reproduce
cd bench && python run.py --run 1 --out ../data/run1.json