I do remember that another member here had a small runt in the litter that his bunny had, and I seem to remember that he did the same as you, putting the bunny back into the nest when the mom would take him out. If she is not feeding him though, you may have to resort to hand feeding him. I do not have any experience personally with babies, but most of what I read online was to allow the mom bunny to take care of things because she usually knows what she is doing.
Sometimes these bunnies are true runts, and just end up being smaller than the rest of the group, and there is also something called a “peanut” baby, peanuts generally do not have a developed digestive system, and cannot live – I hope that I am not upsetting you too much, because I am not an expert on these things so can’t really tell you if it is a runt or a peanut baby, but its important you know that if it does pass away it is not your fault.
I also read online that sometimes it can help to let the small one eat first by cradling the mom bunny in one arm, and bringing the runt bunny up to her and allowing it to latch onto one of her nipples and feed until it is full.