if your hunting your on land or public land shoot whatever deer you want too. a trophy to some is a doe and others a trophy is only a 120 class or better deer.
is it shameful to shoot a smaller buck? absolutely not. if that makes you happy shoot it the first chance you get. and yes meat from a young buck should taste a little better than a buck that is 3-6 yrs old.
personally for me now i let the small bucks walk and wait on a doe to show up when im hunting for deer meat. plus eliminating a doe helps keep your deer herd in check..
there is no right and wrong way in choosing between a big buck or a small buck. if your on your own land or public land then the decision is strickly up to you. now if your in a club please pay attention to your club rules.
and welcome to the site.