neighbors

So, I live in a normal neighborhood . I don't fly my DJI Phantom 2 Vision+ too often in my neighborhood.I always go to a desert area 10 minutes from my house to fly. So I was wondering if I could fly it in my neighborhood? With out police and crazy neighbors attack my phantom. So am I safe or...