Bunnies that age should definitely still be with their mother, is your bun eating and drinking ok by his self? I had a baby bun who was six weeks old (he is a year and a half now) so I can maybe give you a little advice but obviously yours is a lot younger and shouldn’t really be on his own yet so it might be best to check with a vet.
Young buns are super inquisitive so let him come to you, make sure you bunny-proof everything and then just let him out of the cage, sit on the ground and he will eventually come to you to sniff and try to figure out what you are. Try to resist petting him straight away because you might startle him, let him sniff you for a while first. Make sure he has a place to run to where he feels safe if he gets spooked, A decent sized box with a hole for a door will do or even just make sure he has access to his cage, Bramble always used his cage as his safe place and was never interested in anything else I gave him when he was young.
Make sure you are feeding an unlimited amount of a decent baby-bunny food (pellets, not muesli) and unlimited alfalfa hay, no veggies yet he is too young for that. I would get to a vet for a health check asap just to make sure everything is ok, which it hopefully will be. Bramble was fine being away from his mum at six weeks but he had been vet checked before being separated from her and the vet had okayed it, I still took him for a check at my own vet just to be on the safe side mind you because I am a worrier. Also be prepared for the fact that you might in fact have a girl bun, its almost impossible to sex a bunny under 12 weeks old and even then its a struggle – we weren’t 100% sure with Bramble until he was 20 weeks old, neither was the vet in fact we thought we had a girl at 12 weeks because we couldn’t see testicles and its very hard to tell the difference without them.