Do you mean you’ve had him for five weeks, or he’s five weeks old? I’m a bit confused because I can see your bio has an older post about Bailee.
Anyway, if he is five weeks old, that’s far too young to be away from his mother. It’s really not safe to have such a young rabbit outside alone. His diet also needs to be very carefully monitored at that age. It sounds like he’s lethargic from what you’re saying – at that age a bunny is normally with his siblings, so he could be lonely. If all he does is lie about, he could be sick and probably needs to see a vet.
As for lemon branches – a quick google sounds like it’s safe, and there’s two previous binkybunny thread on citrus trees here:
https://binkybunny.com/FORUM/tabid/54/aft/25092/Default.aspx
https://binkybunny.com/FORUM/tabid/54/aft/128528/Default.aspx
But if he’s 5 weeks old, I would hold off on giving him new things until he’s a bit older.
If he is 12 weeks old or more, then I think you can try a small piece of lemon branch.