I have 53 flights with GoPro Hero3BE on my Phantom. ( Yes I keep a log of all my flights)
I have always turned the WiFi on the camera off, no issues so far.
Its really hard to figure out the reason for flyaways, wifi?, too much signal noise in the neighborhood?
The Phantom TX seems pretty weak, and the RX inside the Phantom is maybe an issue too?
Anyway in a week or two I will be replacing the TX/RX chain with a new Jeti DS-16 TX and a Jeti RX.
Even if the Phantom is more or less a toy for me, it would be nice to keep control