I don’t use outlet covers – my bun also does not mess with anything or go near outlets (outlets are in hardwood floor sections, and he runs past them fast to get to carpeting). So it’s really up to your judgement. If you have a pretty nosy bunny, I would do it just to be safe.
You should definitely be letting him out for at least a few hours a day to get exercise – bunnies need a lot of room to stretch their legs even if they have big cages. To bunny proof, start by covering or blocking off access to any cords. Then I would block off access to any books or magazines, because these are irresistible. Otherwise, as long as you’ll be supervising initially, you may not need to do much else. You might decide later that you want to block access under your bed, or that a particular piece of furniture is just too yummy, but you can figure that out as you go. Cords are the most important because even if you’re supervising, it just takes one bite to get through a cord and electrocute your bun – and bunnies move fast and sneak up on things before you even notice!
I think BB posted a great bunny proofing guide in the info section above.