I don't think this will be a problem because we're likely to see a gradual move towards fully self-driven cars where humans still drive but the computer takes over only when needed to avoid accidents. Volvo already has a system that stops you rear-ending other vehicles and BMW has a system that safely pulls the car over if the driver suddenly gets incapacitated. Eventually the most advanced cars will never cause an accident while still being human-driven. From there the leap to let go of the wheel is not that big and I think it will be reasonably easy.