What type of acorns are you looking at? Some are naturally small. We have them from about pea size off of post oak to half a golf ball size off of sawtooth oaks. I'm with Ron, it only takes an odd weather pattern at a critical time to affect them. And that could be 2 years back. Trees seem like a delayed reaction to weather.
We cannot continue to do the same things and expect different results