Preheat oven to 300 degrees. Butter an 8-inch square or 2-quart shallow baking dish; set aside. Toast bread on a baking sheet in oven, turning occasionally, until lightly browned, 20 to 25 minutes. Remove from oven.
In a large bowl, whisk together eggs, half-and-half, pumpkin puree, brown sugar, pie spice, vanilla, and salt. Add toasted bread; cover with plastic wrap, pressing it directly onto surface. Place a plate small enough to fit inside bowl on top of plastic; weight with a large canned good. Let soak until bread is saturated, about 25 minutes.
Transfer mixture to prepared dish, spreading evenly. Bake on a baking sheet until firm and a toothpick inserted in center comes out clean, 60 to 70 minutes. Serve warm or at room temperature, dusted with confectioners' sugar and topped with whipped cream, if desired.