Of course, but the dataset was mostly strings that needed to be cross-referenced with GIS data. Tried every library under the sun. The greatest speed up I got was using polars to process the mostly-string CSVs, but didn't help much. With that said, I think polars was also just released when we were working with that dataset and I'm sure there's been a lot of performance improvements since then.

These only help if you can move the hot loop into some compiled code in those libraries. There's a lot of cases where this isn't possible and at that point there's just no way to make python fast (basically, as soon as you have a for loop in python that runs over every point in your dataset, you've lost).