Its a P4 Pro, so it has obstacle avoidance sensors on all for sides, with visual sensors forward/back, and IR sensors side to side. It's probably braking because of OAS.
This can be happening because there are actually obstacles being sensed and you aren't yet familiar enough with the behavior of this system. Or, the OAS might need to be calibrated (using DJI Assistant 2 and a PC/laptop), falsely sensing obstacles.
Two suggestions: First, upload the flight telemetry from those flights here and others can help see what's going on. I believe the DAT files from the bird itself have information regarding OAS triggers and aircraft behavior/response. That would be the easiest way to get to the bottom of it.
Second, perform a calibration of the OAS with DJI Assistant 2. It's a more involved procedure than calibrating the compass or IMU, but it isn't hard, and is a good thing to go through IMO to get a better idea of how the OAS does its thing.