The larger issue for most quantum key exchange setups is the transition from classical to quantum: you want not to accidentally generate two unentangled photons in the same secret polarization.