It’s good that you’re being diligent with cleaning, and indeed, it’s a bit inconvenient, haha.
Putting him in the cage to try and signal it’s bad to do that will not work. It’s better to bring him to *the litter box* each time this happens.
He’s most likely doing this because he’s trying to establish that out-of-cage territory is his territory, and constantly being put back in the cage is making him want to reinforce that every time he’s let out. I suggest a litter box outside his cage, which you can bring him to when he pees outside; this way, he has a place to do his business in the out-of-cage territory. Neutering will help to lessen the marking, but he needs to feel secure that the area is his territory, without peeing places he shouldn’t.
Since you’ve had him a month now and he knows the environment better, he is much more aware that while in the cage, he’s being barred from a lot of his territory, so being in there for hours on end may be getting more and more difficult for him. Even if, with the cage open, he spends time in the cage, it’s the idea of access and control over his space.
The answers provided in this discussion are for general guideline purposes only. The information is not intended to diagnose or treat your pet. Seek the advice of your veterinarian or a qualified behaviorist.