It is actually very important to know a deer's age. If you are a landowner who is trying to boost your genetics for big bucks, you can look at the deer you kill and see what age they are. You might find that you have small built 3-4 yr old bucks with small or deformed racks. This means that you have bad genetics being passed on. This would lead you to take out a few of the smaller bucks so the bigger, more dominant ones will pass on the better genetics.
Also, if you get a general concensus on the age to size (body and rack) of a deer, you can be more informed about the deer you do shoot. If you have 2 yr old deer normally being 8-10 points, you might want to start holding off on shooting those and give them that extra year to grow into something only a few of us have gotten a chance at. If you are seeing that your deer are underweight for their age or have small rack growth it could lead you to plant different food for them.
Finally, if you know what 1 1/2 to 2 yr old deer in your area look like, you can also judge the strength of your deer herd before season (scouting while they are in velvet)
Shooting the first/small bucks that you see often is what makes public land so bad to hunt. Just because something is a nice 8-10, it doesn't mean that you should shoot it to get another deer under your belt. If that is a 1 1/2 old deer....you just shot what would've been a monster in two more years.
If you are shooting just to get some meat, take out a few does....that always helps the herd and makes hunting around the rut more fun because the bucks have to work harder for the fewer does...this also means that the better genetics will be passed on too.