With a new environment, bunnies need time to understand the changes and recognize that you aren’t a threat to them. You should not try to actively seek out petting him, that will make him want to stay away from you because every time he comes near, you reach for him.
Sit on the floor at his level and just be still and let him get used to your presence. After he spends some time around you and you get him to realize that you aren’t a threat, he may begin to trust you more. Try feeding him pellets, veggies, treats out of your hands so that he associates you with something pleasant.
The main thing is patience. Some bunnies take months or longer before they feel comfortable. You have to view things from his perspective and understand that you are a massive, scary, potential predator (until you prove otherwise to him). His trust will come with time. They are prey animals so they are naturally inclined to be afraid of us.