Interesting premise, especially given Kal-El's long life expectancy under a yellow sun.
If Jor-El's sending the baby to Gallifrey instead of handing him off to the Doctor in person, he's likely to end up with the fourth Doctor, who spent much more time at home than the other incarnations.
Unless the Doctor is off-world and the spaceship gets picked up by someone else...
(Romana wouldn't be so bad, but the Rani, the Master, Rassilon, Omega... ::shudder:: )
