You are correct, the FAA only requires notification not permission. You do not need to provide your registration number to anyone.
Only some areas do require permission,
"Model aircraft must not operate in Prohibited Areas, Special Flight Rule Areas or, the Washington National Capital Region...