I think the issue is more because of the way you are holding him on your chest. Depending on his history, he may have been held on his back a lot (breeders tend to do this a lot). If he is fairly vertical when on your chest, he may be almost anticipating being flipped and kind of slip into the position easily. Or if he is pushing back, he could push himself into this position.
Rabbits generally love their ears being petted, but most rabbits prefer to be petted while sitting on the ground.
I recommend petting your bun while sitting on the ground with him, and if you have to carry him, make sure he’s held in a position where he doesn’t tip back past vertical. Hold him firmly so he can’t wiggle around.
.
.
.
The answers provided in this discussion are for general guideline purposes only. The information is not intended to diagnose or treat your pet. Seek the advice of your veterinarian or a qualified behaviorist.