Learn to fly in Atti mode. When that happens just switch modes and keep in line of sight. Calibrate the compass in an open area. I fly my f550 in front of my condo unit from time to time and had issues with it in gps mode. I had to reach up and grab it once because I had no stick input then quickly switched to Atti mode to control it. It happens pretty quickly but if you realise it's drifting away, switch to Atti to disable the compass.