I'm quite sure that the FAA would like that to be the interpretation. 14 CFR 91-131 was clearly the inspiration for their Class B statement. That is clear from their notice of interpretation of the special rule in 336, published in 2014, which was a clear attempt to circumvent the restrictions on further regulation of model aircraft. It's FAA weaseliness at its best, arguing that they are only prohibited from promulgating further rules that only apply to model aircraft, but that they can apply all other aviation rules, current and future, that apply to manned aircraft:
Section 336 also prohibits the FAA from promulgating “any rule or regulation regarding a model aircraft, or an aircraft being developed as a model aircraft” if the following statutory requirements are met:
• the aircraft is flown strictly for hobby or recreational use;
• the aircraft is operated in accordance with a community-based set
of safety guidelines and within the programming of a nationwide
community-based organization;
• the aircraft is limited to not more than 55 pounds unless otherwise
certified through a design, construction, inspection, flight test, and operational safety program administered by a community-based organization;
• the aircraft is operated in a manner that does not interfere with and gives way to any manned aircraft; and
• when flown within 5 miles of an airport, the operator of the aircraft provides the airport operator and the airport air traffic control tower ... with prior notice of the operation....
P.L. 112-95, section 336(a)(1)-(5).
Thus, based on the language of the statute, we conclude that aircraft that meet the
statutory definition and operational requirements, as described above, would be exempt from future FAA rulemaking action specifically regarding model aircraft. Model aircraft that do not meet these statutory requirements are nonetheless unmanned aircraft, and as such, are subject to all existing FAA regulations, as well as future rulemaking action, and the FAA intends to apply its regulations to such unmanned aircraft.
The last sentence refers to Part 107. They go on to say:
Congress directed that the FAA may not “promulgate any rule or regulation regarding a model aircraft, or an aircraft being developed as a model aircraft” if the aircraft is being operated, or being developed to be operated, pursuant to the five criteria enumerated in the statute as described above. P.L. 112-95, section 336(a). In other words, Congress has restricted the FAA from promulgating regulations, from the date when the statute was enacted, specifically regarding model aircraft that meet the terms of the statute.
However, the prohibition against future rulemaking is not a complete bar on rulemaking that may have an effect on model aircraft. As noted above, the rulemaking limitation applies only to rulemaking actions specifically “regarding a model aircraft or an aircraft being developed as a model aircraft.” P.L. 112-95, section 336(a). Thus, the rulemaking prohibition would not apply in the case of general rules that the FAA may issue or modify that apply to all aircraft, such as rules addressing the use of airspace (e.g., the 2008 rule governing VFR operations in the Washington, DC area) for safety or security reasons. See 73 FR 46803. The statute does not require FAA to exempt model aircraft from those rules because those rules are not specifically regarding model aircraft. On the other hand, a model aircraft operated pursuant to the terms of section 336 would potentially be excepted from a UAS aircraft certification rule, for example, because of the limitation on future rulemaking specifically “regarding a model aircraft, or an aircraft being developed as a model aircraft.” P.L. 112-95, section 336(a). The FAA interprets the section 336 rulemaking prohibition as one that must be evaluated on a rule-by-rule basis.
Credible effort, but not adopted, so far, in practice. With regard to airspace, they then tried to argue as you have:
Generally, if an operator is unable to comply with the regulatory requirements for operating in a particular class of airspace, the operator would need authorization from air traffic control to operate in that area. See, e.g., 14 CFR 91.127(a), 91.129(a). Operations within restricted areas designated in part 73 would be prohibited without permission from the using or controlling agency. Accordingly, as part of the requirements for model aircraft operations within 5 miles of an airport set forth in section 336(a)(4) of P.L. 112-95, the FAA would expect modelers operating model aircraft in airspace covered by §§ 91.126 through 91.135 and part 73 to obtain authorization from air traffic control prior to operating.
However - that's not how it is being interpreted for other classes of controlled airspace - airports are readily agreeing to recreational flights within 5 miles and in controlled airspace. The FAA's own guidance app does not warn about airspace classes, even though it will list them - it just tells you to contact airports within 5 miles, stating "Warning, Action Required" rather than "Flight Prohibited".