Don’t be scared! If you want to make him happy, he will be happy.
You can bring something with you from your sister’s house that smells of that place, the way you do with puppy dogs, if you’re not bringing his whole cage because then he has his home in your home.
When a bunny comes to a new place, it wants a hidey-place. That can be a cardboard box turned upside down, with door-holes cut in it. If a bunny has a safe place to withdraw to if he feels the need, he will feel more confident and dare do more. If you don’t have a cardboard box, a fot stool might do, or anything else that has like a “roof”, bunnies like to sit under roofs because they fear predators that come from above.
And the general rule is let the bunny come to you. He seems like a loving little one, and you seem like you have the right instincts to become a good bunny mommy. Please keep us posted on your progress! And (hint, hint!) we like bunny pics here…=)