It seems like this might be pretty normal. It makes sense that the longer and harder you fly, the hotter your motors are going to get and the less efficient they're going to become. And they less efficient they are, the slower they'll be to respond to the main board's commands. So the more the Phantom is going to jump around. Probably amplified by the fact that the standard motors don't have the most efficient barrings in the first place. Just a theory. You might try giving it a rest between batteries... let the motors cool down a bit... and see if that makes a difference. If so, you might then consider switching to more efficient barrings and/or motors.