Every bunny is different; give him a chance to settle in and don’t rush the process along. Simply sitting with your rabbit in a rabbit proof room or pen and letting him come to you (and that may take awhile) is recommended. Ignoring him (not neglecting him, but simply reading a book or sitting quietely in his prescence can have wonderous effects; bunnies may be shy, but they are also notoriously nosey, and will often come up to you and sniff you or even bump you with their nose if you just take the time to get to know them on their level (down on the floor). Have a little healthy treat for him ready if he does come over to you, but don’t be disturbed if he runs away. As for litter training, have you put hay in his litterbox? I find that encourages them to use the litterbox for its intended purposes. Don’t worry about him messing on the hay–they generally will poop on one end and eat from the other. I also put the water bottle on the same side of the pen as the litterbox (like hanging over it, so to speak). This encourages him to drink and pee at the same time.
I just adopted a new bunny from a shelter but already have a bunny who is litter trained. Simon, my shelter bunny, didn’t come litter trained but has been here a week and seems to have caught on; perhaps being with the second rabbit helped (although they had to have intensive bonding sessiosn to be kept together in the same pen, though). Sometimes having two litterboxes helps, and not placing bedding on the floor of the cage (I find that would confuse a bunny as to were to go).
Your bunny is still new and it will take some time for his personality to come to the forefront; on the other hand, he may simply be a nervous bunny who needs extra time to trust you. Remember, they are prey animals and not like cats or dogs–it takes them time to trust you.