Uh, then don't get it. It's not like they aren't releasing a smaller phone called the iPhone 6 as well. Plenty of people with small hands are big fans of Samsung's Galaxy Note series, so maybe not everyone else in the market hag the same ergonomic preferences you do.
Android's much more fragmented and yet hasn't had any trouble making gains. As much as Apple and Apple-boosters have made of fragmentation as some great cost, actual market experience hasn't really born that out in terms of selling handsets.
It does make things more difficult for developers, sure.