One big difference though is jitter tolerance. To some extent, dropping visual frames will be seen as a reduction in average frame rate. For example, if scroll stutters, people may brush it off as "oh yeah the machine's loaded right now" and forgive and forget. However, the tolerance level of audio glitches is much lower (I think for most, but certainly for me). So the leeway for a computation to bleed over from one frame to the next is far less than for video.