Hello, welcome, and I’m sorry to hear you’re having this issue!
Neutering your rabbit should definitely help, but do remember that hormones can stay in a rabbit’s system for a week or two after the procedure, so the results may not be immediate.
Is there any other space you can lead him to where he can’t see you picking up after him? If you can lure him somewhere where you can block his sight of you picking up after him, this may help.
Another thing you might try to do is to give him a healthy treat before you pick up, while you pick up, and after you pick up, to distract him from feeling territorial. Similarly you could give him his daily pellets when you’re going to clean up – this may or may not work.
Also, since he’s so new to your home, he may be freaking out because he’s still trying to outline his territory. It may be the best idea to limit his out time to an x-pen attached to the enclosure or to simply have him in his enclosure at all times until he is neutered. Security will make him feel less territorial and more comfortable using the box.