I think Rye actually does handle this mostly correctly (as the sibling comment said). I got through some of it here: https://github.com/mitsuhiko/rye/issues/671. I think it's very close to what I actually want (maybe not what Armin wants with multiversion).