I started out around the age of nine with a Ruger .223 (Mini 14). With the right scope and quality marksmanship, it will down a deer with no problem. Knockdown power is fair due to low grain bullets, but it has barely any recoil at all and has exceptional range.
When I got older (about 12 y/o), I moved up to a Remington Model 700 .25-06. This caliber is flat shooting, has plenty of knockdown power and carries good range. Hornady makes a 117 grain bullet for this caliber and I have found it to shoot the best.
Of course you could always go with the classic .30-30. It has a little more recoil and less range, but my sister used one when she was 14 y/o so it can't be that bad on a youth.
Good luck with your choice and in the field!