Melt the butter in a skillet over medium-high heat. Add the brown sugar and cook until melted. Add the citrus juice and cook, whisking, until a smooth sauce forms. Mix in the spices. Add the bananas and turn to coat. Add the brandy or rum, tilt the pan slightly, and very carefully ignite with a long match. Cook until the flames die out, about 3 to 4 minutes. Serve warm over ice cream.