He will not grow to not like you
The first day you had him he was probably subdued by all the new impressions. It’s a good instinct to lay low the first time in a new and unknown environment. For many buns, this stage lasts a week, but Harlow seems to have snapped out of it in a day only. If he calms down when you talk to him, that’s really great.
The best thing to get a bun to trust and like you is to do nothing but sit with him when he has out-time. You can read a book or a magazine. Don’t touch, don’t try to grab – don’t even look at him, really (predators eye their prey before they pounce). Rabbits are very inquisitive and he won’t be able to resist coming to check you out sooner or later. But he will probably want to explore his new environment first. You can lie down on the floor and let him use you as a jungle-gym, that’s often irresistible for a young bun.
You can help him feel more comfortable in his new environment by providing one or two card-board houses, i e cardboard boxes with exit and entrance holes cut in them (they tend to want two holes or they can feel trapped. A bunny burrow always have two exits.) If a bunny has a safe place to withdraw to, it often feels more courageous.
To me, the way you describe his behavior sounds very normal and very promising.