I've found that on the highway that someone in an adjacent lane is most likely to make a sudden lane change at very high or very low speeds. So at 5mph, from my experience, you're far more likely to encounter situations where you need to be paying attention than not when you're driving at low speeds on a highway.
So yeah, using your phone on the highway, in really any situation unless its park-and-sleep traffic, is kind of a dick move.