If everyone on your block (and the rest of your ISP) is streaming Netflix at the same time it's more likely to be slow either because your ISP has effectively capped Netflix traffic upstream by not adding more peering connections [0] and/or is not taking advantage of Netflix' freely provided cache[1] system.

0. http://gigaom.com/2013/06/17/having-problems-with-your-netfl... 1. https://signup.netflix.com/openconnect