Could be several things!
1) bad motor bearing
2) failing speed control
3) loose connection
4) shell cracks ( where motor mounts )
5) failing GPS signal
6) compass issues
Is first try to diagnose which motor it is coming from. Look for stress cracks under motors first, if all good, then check all connections and solder joints for problems ( look closely, sometimes things LOOK plugged in, but are just out enough to cause issues! If that looks good, manually spin each motor ( see if all spin the same ( not gritty, or tight ) then, If you have a thermal temp gun, pin craft down ( weight it down so it cannot lift off ) run full for a couple minutes, check motor temps, if any are abnormally high temp ( bad bearing ) if no move onto possible bad or failing esc ( multi-meter ) to see if you are getting 5.vdc at esc power leads! Ect.
May also just need a fresh ADVANCED IMU CALIBRATION!
Craft STABLE, and LEVEL, in ALL directions , then perform calibration!
Once finished, do a fresh compass calibration, go test!
I don't know what all you've tried yet, but those are good starting points!
I had same problem with a p2v, was combination bearing going bad, and stress crack in the shell!
Let us know
J Dot