He may be a little perplexed over this whole new situation. I don’t think you are doing anything wrong. I think you are doing great, actually, you’re letting him set the pace. You could perhaps make a hidey-house for him of a cardboard box, make two holes so he can enter and exit and not feel trapped. maybe he’d like a house like that and feel safer and want to come out and explore more. Also reward him with treats if he approaches you on his own. He needs to learn that you are a resource to him, not a threat.
It is possible that he “misses” bunny companionship since he’s been used to that for such a long period of his life.
Your description reminds me of my bunny Bam. It took a vey long time for him to warm up to me, he sat under the coffea-table more or less all the time until he suddenly changed, almost over-night. He then became a very cuddly bunny. He likes to explore, run in his tunnel, roll his treat-ball and hang out with his humans and his dog.
At age 4 few rabbits play a lot and some never play regardless of age, so that’s not sth to be worried about. Nibble stick is good, you can also give him fresh apple twigs or willow, rabbits often like to chew the bark off those.