Some rabbits are simply so Royal that they just can’t lick YOU directly. Take it as a compliment that he is grooming you by proxy (especially fi he is doing it while you are watching, he is allowing you to view the grooming of something that is [to him] an extension of yourself).
My girl rarely grooms me, and by me and I mean clothing I am wearing at the time. She is move likely to lick whatever I’m seated on, so the sofa or floor cushion. She has groomed my jeans twice, last time for quite a while too. I was shocked at just how wet a rabbits tongue really is! (especially since I’m used to a cats usually dry tongue).
To a rabbit the groomer is the submissive bunny, and the groomee is the dominant one. So the groomee doesn’t HAVE to ‘return the favour’ that he licks anything that smells of YOU and not him is a massive compliment. You’re bunny loves you and he loves what you do (I assume he is more likely to groom things that smell of you after you have been petting him right?)