I have high end scopes on both of my rifles. Bought them obviously before I got married. I shoot paper with them at least 3 or 4 times throughout the year, just to make sure nothing has changed. And also, I don't pull the trigger a lot during deer season, and it'll knock any rust off that's built up on me in the last couple months. Luckily our place is set up where I can do that and not affect anything as far as hunting.
With all this being said, I wounded the only deer I've shot at this year.
It happens from time to time, but not 5 times a year! I hope not anyway!