"If I have to shoot 20 times in a hunting season with a 6 deer limit I should probablly take up golf or something else." About fell out my chair when I read that, will have to remember that line - especially the way I play golf.
As someone else has already said it -- it's all about shot placement. You put a shot through both lungs regardless what caliber and the deer's not gonna be running very far or very long. Who knows - you may even catch a rib on the way in helping the bullet to expand and do what it's supposed to do. Double lung also damages very little meat. Save the head shots and go for a more "sure" thing. But that's just my humble opinion.
For recoil - see the thread I started on this very subject. I zeroed in a Remington 81 taking 12 shots in about 15-20 minutes. Shoulder was bruised and sore for days. Asked the guys about a recoil pad and got a great response. Ordered the recoil pad directly from their website and it's already at the house. LimbSaver : Products : Precision-Fit Recoil Pad