RTH (return to home) is activated by switching off the controller (the one in your hands). Putting it to off.
You have no Phantom mode in the software, you have an FC40, you will install the NAZA assistant. NAZA mode.
The activation is for the IOC on the left switch on the controller. That will activate Course lock and home lock.
You can activate failsafe on the right switch if you want, but rather activate manual mode, but this mode is the absolute LAST mode to use, in case of a complete loss of control of the bird with any anything else, including switching the controller off.
Please read the manual for their description. It is very important you read through the manual entirely and these forums (and others). It took me a solid week of reading before I got mine, 3 hours to setup and configure the software, understanding what and why I'm doing stuff, IMU, compass, magnet, etc....
I got around 1.5 hours of flight time, and I'm still doing boxes and figure 8's, and I still get the controls messed up from time to time. This is an expensive hobby, and worst case can cause injury and god forbid death.
Get to know what you are doing inside out.