I also keep mine with my p3p and have had no problems.
I have 2 iPad minis, one with this type of cover, and one with a wrap around gorilla type cover. The heavy duty gorilla cover is so hard to remove that I never fly with it, always use the folding one.
It does leave dust lines on the screen...