No, it won't. Game mode reduces latency on the order of 2-3 frames, so a 0.1sec or less. The issue is that the tuner has to consume a complete data frame before it can start displaying, since it needs to get a keyframe for the codec to start working.
Hmm, I think I may have noticed that issue switching between HD channels at times, but SD channels maybe don't have the same issue?
I guess the time before a key frame appears can vary from up to a second to as low as 0, depending on when you happen to switch. It could be that a few frames, or 0.1 seconds becomes significant at that point, though you're right that it's likely to make little difference. I was not aware of the key frame issue until now, though; thanks for enlightening.
Basically the way modern video codecs work is that they only fully encode every Nth frame, where N is some smallish integer - 15 or 30 is common. From one keyframe to the next, they essentially encode a diff from the prior frame. The idea is that most of the regions of the frame have very little change over short timescales.
It's a tradeoff. More keyframes means faster seeking (when you seek in a video, it has to start from a keyframe too, for the same reasons), but larger data rates for the same image quality.