I don't calibrate every flight. I only calibrate after firmware updates, if I'm flying more than about 150 miles from last take-off or if I've installed or removed any hardware/had the case open. It's easier to do a bad calibration if you calibrate before every flight, in my opinion. However I always do a 30 second hover test at the start of every flight and watch for odd behaviour.
If the aircraft isn't asking for a calibration then it feels everything is ok, obviously. But I'd do that hover test - close in and about 10ft up - and make sure it's a tight hover with no circling or other odd flight behaviour. The fact you are getting recalibrate warnings leads me to suspect you're recalibrating in an error with some ferro-magnetic interference from somewhere. I'd go to a different area, nice and clean, no metal (including underground), no concrete, railings, drain covers etc. When you get a good calibration there personally I'd not bother doing another one unless the aircraft asks for one, or you notice odd flight behaviour during a hover test, or you update firmware or the other things I mentioned above.
Just my opinion, I know there are many who advocate a calibration before every flight. I just feel there's more risk of a bad calibration by doing that, rather than leaving it with a previously good calibration locked in (mainly from bitter experience of flying after calibrating in a stupid place back in the early days!).