Preheat the oven to 350 degrees F. Place 6, 1/2-cup custard cups in a baking pan. Place the first 4 ingredients in a small pot and set over medium heat. Bring to just below a simmer, and then turn off the heat.
Place the egg yolks and granulated sugar in a medium bowl. Whisk until the yolks are pale yellow and the sugar is completely dissolved. Very, very slowly dribble and whisk in 1 cup of the warm eggnog. Whisk this mixture into the remaining eggnog in the pot. Pour this custard into the cups. Set the pan in the oven and pour in enough hot water to come two third’s of the way up the sides of the cups.
Bake 35 minutes, or until the custard is just set, but still jiggles. Remove the cups from the pan and cool to room temperature. Cover and refrigerate for 4 hours, or overnight.
Place an oven rack 6" below the broiler. Preheat the broiler. Set the crème brûlée on a baking sheet. Sprinkle each with 1 Tbsp. of the brown sugar. Broil until the sugar melts and coats the top of the custard, about 60 to 90 seconds. Remove from the oven, cool a few minutes to allow the melted sugar to set, and then serve.