I accept your criticisms of SMS for authentication (I recently switched from SMS to the Android app), but I like two factor better than the approach you describe. If I log on to GMail from a public computer at a library with a keylogger installed, they will obtain my password but not enough to log in as me after I have signed out. Under the scenario you describe, I'd also type the answer to a challenge question, and they'd have both the password and the answer to the challenge question. That would leave me in a significantly worse position.