My bunny has had the same thing for as long as I can remember. What are you feeding your bunny? Sometimes if a rabbit eats a mostly-pellet or a lot of sugary foods (fruit) diet they can have softer poops that stick to their bum. Slowly weening him down to 1/8th cup of pellets and eating mostly timothy hay might help to firm things up.
In the mean time, however, sometimes using alcohol/scent-free baby wipes can get the poop dislodged, but if you feel iffy about using baby-wipes even a rag or paper towel with warm water could help. If you’re having a LOT of trouble, you could try soaking his bum in about an inch of water in an old litter pan or similar box structure. I say this is a last resort because many bunnies don’t like the water, but if it’s shallow enough to just get his bum soaking it shouldn’t be too bad.
When Meki is fidgety and doesn’t let me flip her to get the poo off, sometimes I’ll just distract her by having someone pet her face and cheeks while I go ’round back and lift up her butt-skirt with one hand and get to the poo with the other. If all else fails, it may be best to get a buddy so one of you can hold him upside down and calm him while the other gets the poo off. With two pairs of hands working on it, it should go by faster for him to not be in such a position as well.