Answer 1, the double standard.
Answer 2, the penis is linked to aggression, and to show it, though it could be meant as offer, often will be read as threat. It would be hard to hurt somebody or invade somebody’s body with a breast.
Answer 3, we have fetishized the naked body and especially the genitals.
There may be other answers. It’s an interesting question, for sure.