From what I understand, they could be doing that for a number of things, such as marking their territory.
For me, i’ve had my bun for only two-three weeks. At first, she’d poop everywhere in the cage, but only urinate in the litter box. As time rolled on, she slowly started pooping more and more in the litter box exclusively. It probably didn’t help that I got her a new cage so that she started re-poop-marking. Now, she mostly goes exclusively in the litter box, but may around the rest of the cage if I had just cleaned it. At least when I let her around the house, she won’t go to the bathroom. She’ll run straight to the cage to go!
If it is a territorial thing, it could be due to the fact that it’s a new environment and would fade with time. The more frequently you clean it (and using deodorizers) the longer they’ll do it until it stops. If they are not spayed/neutered, that could also result in marking around their space and would not improve with time. My bun doesn’t like being picked up too much. What I do is pick up the poop and just put it in the litter box, in hopes that they’ll associate the litter box with defecation. So far, it’s been working. Putting hay in the litter box may help too. I keep my buns hay feeder away from the litter box, but I put at least a little hay in the litter box as a treat.