It will only crash, even in lost signal, due to equipment or pilot failure.
Proper planning and preparation negate the issues associated with RTH.
The likelihood of RTH equipment failure is about the same as the likelihood of total quad equipment failure.
I suppose it's all about acceptable risk margins. But then again, as a 26 year old, my margin might be a bit different than some of the other folks around here. I try my best to maintain my bird, plan in advance, and not take impulsive "I just need to get that shot" risks, but at the same time I fly over dense residential quite frequently. Likelihood of it falling from the sky is low, maybe 1-100, likelihood of it hitting a human during falling (at least where I fly) even lower, maybe 1-500, so total likelihood of any given day hitting a human, negligible. I have more chance of getting in a car accident on my commute to work. Doesn't stop me from driving.
That all said, when I test new equipment, flight modes, or fully autonomous features, I do so over parks and other "green space" around me, not over houses or people.