If my smartphone is the remote then no-one can use it when I'm not at home. So instead you have to set up every smartphone as a remote
That's not how it works. Anyone connected to your Wifi, is "set up" to use the Chromecast and can stream from any number of apps like YouTube, Netflix, Google Play Music and Movies, Pandora, and now even Rdio and Vudu.
They still need to set up those apps on their phone. They need to log in to Netflix, Amazon Instant, etc. etc. even if they never watch content on their phone.
That's not how it works. Anyone connected to your Wifi, is "set up" to use the Chromecast and can stream from any number of apps like YouTube, Netflix, Google Play Music and Movies, Pandora, and now even Rdio and Vudu.