Normal logic seems to go out the window when rabbits are involved… lol.
A few things to clarify:
1) It does not matter that they are siblings. When they are reintroduced, they will regard each other as strangers. So if unneutered, they most likely will fight.
2) It does not matter if one is currently in a group and one is not. Every rabbit relationship is unique. Some pairings work, some do not. Just because brother is with a group does not mean he will be easier to bond. And just because your bunny is single does not mean he will be more difficult to bond.
3) Size certainly does not matter. There are some very gentle giants and some downright nasty dwarfs. Just because your bunny is smaller, does not mean he is in danger simply because of his size, and this also does not mean that the larger bunny will be more dominant.
If the brother is already in a group, why are you looking to split them up? Is the sister separated from the remaining boys? If so, and she’s by herself, I might consider taking her and introducing them after they are both altered. Male-female bonds tend to be easier, but it is (of course!) imperative that all buns are spay/neutered.