Thanks to all of you whjo answered, but it seems I'm still a bit in the dark here.
On the one hand, some say as long as you don't take off or land within 50 metres of people, you're OK, but then someone quotes this:
"The Air Navigation Order defines a congested area as being 'any area of a city, town or settlement which is substantially used for residential, industrial, commercial or recreational purposes'."
Basically, this seems to mean you couldn't even fly over a tiny hamlet, since that would be aplace 'substantially used for residential purposes'.
Does that mean any video you see on the Web of a village in the UK (and I'm sure there are lotson the Web) is the result of breaking the law? Seems very harsh.