Some bunnies are territorial and with those you should avoid cleaning the cage while the bunny is in it etc, My bunny Yohio growls sometimes. He used to growl a lot more and he would box at me too. I think he’s gotten used to me touching “his” stuff now. He would growl, then immediately demand pets, so I didn’t really feel a need to try and stop his behavior. I just say “No growling at mommy” to him. I think it’s a bit of insecurity on the bunny’s part.

If he is not neutered, that would probably help.

I would make sure he is out of his enclosure when you move his litterbox. He’s warning you to respect his space and that is what you should do.