One downside I see to this is it being illegal to use a phone while driving, but interacting with your car stereo is fine.

Now I'm wondering if any car stereos have four-pin aux inputs to send headset button inputs or microphone audio back to the phone.