A cage of some type (preferably a dog xpen or wire grid large condo) will be necessary to provide the rabbit that feeling of security. Some people choose to have a rabbit free range in the house, but that is later on down the road, after neutering, after the rabbit is an adult, and once he’s been around for awhile (a year or more). You need to know that the house is fully bunny-proofed and he is trustworthy before you go cage-less. And many people keep the cage (again for the rabbit’s security, having his “own space”), but leave the door/gate open 24/7.
Rabbits are smart and if his space is set up in a way for him to succeed, he will. Neutering is vital and providing a large litterbox with appropriate litter and hay will help him grow accustomed to using the box for this purpose. Age can make a big difference; the younger the bunny, the more time it may take for them to catch onto the litterbox. Older rabbits are pretty good with it.