sounds like you made a great ethical/personal decision by passing up what may have been a momma Doe with yearlings and I applaud you for showing this much respect to the animals,, many hunters approach a hunt wanting to harvest any game animal they see which is fine as long as they hunt following the Laws (Federal & State rules and regulations...)
Either way, as long as it was within the law and legal for you to harvest a Doe, It's totally your decision on what you choose to harvest and what you choose to let pass,, many fawn/yearlings fall prey to predators anyway but there are many other factors that come into play when deciding whether to harvest Bucks or Doe's,, young or old... Food supply, Habitat, carrying capacity, Herd size, Health of the herd, Buck to doe ratios...
We as hunters are simply the tools our State's use to try to establish, healthy, balanced wildlife...
Good Luck