Fair enough, it’s totally possible that’s what they meant. But the complaint of “every package assumes you loaded the whole dataset in memory” seems to imply the package just naively reads the file in. I mean, if the package was mmapping it, they probably wouldn’t have had much trouble with memory enough for it to be an issue they’ve had to complain about. Also, you may not always have the luxury of mmap()’ing, if you’re reading data from a socket (network connection, stdout from some other command, etc.)

I don’t do much python but I used to do a lot of ruby, and it was rare to see anyone mmap’ing anything, most people just did File.read(path) and called it a day. If the norm in the python ecosystem is to mmap things, then you’re probably right.