Hello and welcome to Binkybunny!
Do you know why the person told you that you should be picking them up and handling them everyday? I’m not exactly the most familiar with babies but from what I know, it’s not necessary to pick them up everyday at all. Rabbits generally don’t like to be picked up so they could be scared of you because you are doing that.
You can sit or lay with them and allow them to come and sniff you, rather than reaching out to hold or pet them. You also need to keep every single one of them separately, including the parents. The parents need to be spayed and neutered to prevent any accidents again. The babies may be a little too young to be sexed properly, but you still need to separate them because unfixed rabbits will either fight or mate, both of which are undesirable. When they are of the right age, you will have to spay/neuter them as well – if you would like to keep all of them, then they will have to go through a proper bonding process to ensure there are no fights when they are kept together.
It may seem overwhelming but it’s not easy taking care of rabbits (what more so many of them). All the best and do ask if there are any questions that come along the way!