Prepare the remoulade by combining the first 9 ingredients in a small bowl. Cover and refrigerate until needed.
Preheat the oven to 425 degrees F.
Line a baking dish with parchment paper. Set in the halibut. Drizzle and sprinkle the fish with the olive oil, lemon juice, salt and pepper.
Bake for 15 minutes, or until cooked through. Plate the halibut and serve with remoulade sauce.