The pooping habit is him marking his territory. ALL rabbits do this in new areas but it often subsides in a couple of weeks. Of course, if you wash or shampoo the areas, the pooping may return for a short stint.

Rabbits will chew, and sometimes ingest corrugated plastic, which isn’t overly healthy for them. If it is only ‘dry’ waste, I’d suggest using cardboard. He’ll wind up chewing it up so you’ll need to replace it from time to time but it won’t hurt him. Having hay in a litter box isn’t an issue but I’d suggest finding a way to secure the hay box to the walls. Some bunnies are diggers and just like to pull hay out and throw it about. It could also be that the rabbit is bored so maybe some rabbit safe toys will work.  If you don’t mind the mess, I’d suggest a telephone book.  Diggers seem to love to tear them up.

Above all, I’d say his demeanor will calm down a little when he is neutered. You should do so as once they get to the adolescent phase, the marking and other things can increase.

I would also suggest getting him neutered because a lot of bunnies will get much better litter habbits when they are neutered, not all but quite a few. I used corrugated plastic and my bunny just chewed it up so I would use cardboard.

I second getting him desexed, my male bun used to poo EVERYWHERE until i got him done about a month ago and now it has magically stopped hurrah! as it is a territorial habit neutering should drastically reduce this habit (it will take a few weeks after the procedure though so dont be too worried if he doesnt change immediately)

In addition to the poop marking, some buns (males in particular) will start to spray urine to mark territory. Picking up poop pellets is easy, cleaning dried urine off walls, not so much. He will be calmer after neutering and won’t have any urge to mate, so while it may seem barbaric to you, since he’s not ALLOWED to mate, it’s actually the best thing for him to remove those hormones from the equation.

I second the neutering.
Aspen litter is excellent, so I’d continue to use it in the litter pan.

I would suggest a hanging hay rack. It will make him unable to poop in it, but he can pull the hay down to eat. You can even hang it over the litter box, so when he eats his hay he will poop at the same time but already be in the litter pan so it will contain it.

Welcome!!

Corrugated plastic is very safe-you can get it at home depot/home hardware, rona places like that.
Yes-hay in the litter box is the way to go-that will help him to poop in the litter box, but neutering is what really will make the difference-as well as keeping his area tidy-the longer the poops are on the ground, the more that is normal. If you pick them up and put them in the box right away he’ll get the picture.

I have coroplast surrounding my bun’s cage and it works very well to keep things inside! I would suggest putting it on the outside of the cage, because when my bun is out to play, he does like to chew on the coroplast every now and then – so if it was on the inside of the cage, I’m afraid he’d chew on it quite a bit.

A different litter box lining you could look into is wood stove pellets. They absorb better than aspen shavings and are cheaper. I used to use aspen and then when I found out about wood stove pellets, I switched and I love them! 

A hay rack would be a great thing to get. Placing the rack above his litter pan (on the side of the cage) keeps the hay off the floor and he can’t soil it.

My rabbit isn’t fixed either. He never pees outside of his cage, but he does leave droppings every now and then. I found that having him run around with his cage door open allows him to hop in and go potty, then come back out to play, and this has drastically reduced the amount of poop he leaves outside of the cage. So if you don’t already do that now, you could try that.

Good luck!