Squeeze the juice of one orange and one lemon, making sure there are no pips. Chop the remaining orange and lemon. Add the juice and chopped citrus to a pitcher. Mix in the apples, nectarine, sugar, cloves, cinnamon, vanilla bean, red wine, and brandy, and let sit in the refrigerator for at least 4 hours, overnight is best.
When the party is about to start, add a few ice cubes to the glasses, top off the sangria with some sparkling water.