Heat oven to 350 degrees. Butter a 9-inch springform pan; set aside. Spread hazelnuts on a baking sheet. Bake until fragrant, about 10 minutes. Rub warm nuts in a clean kitchen towel to remove skins. Let cool. Place nuts in the bowl of a food processor, and pulse until medium fine, about 30 pulses.
Transfer nuts to the bowl of an electric mixer fitted with the paddle attachment. Add 1 1/2 cups flour, 1/2 cup granulated sugar, light-brown sugar, 1/2 teaspoon cinnamon, and 1/2 teaspoon salt; mix until just combined. Add butter, and mix on low speed until crumb begins to stick together, 2 to 3 minutes. Press 3 cups of crumb into the bottom of the prepared pan and about 1 1/2 inches up the sides of the pan to form crust. Set remaining crumb mixture aside.
Transfer crust to the oven; bake until it appears to be set, 18 to 20 minutes. Set aside.
Slice plums in half, and remove pits. Slice into eighths. Scatter 3/4 cup raspberries and sliced plums onto cooled crust; set aside.
In a medium bowl, whisk together 1 tablespoon flour and 1/4 cup plus 2 tablespoons granulated sugar. Whisk in egg, egg yolk, heavy cream, milk, 1/4 teaspoon cinnamon, 1/4 teaspoon salt, and nutmeg (optional). Pour custard over fruit; sprinkle with remaining crumb mixture. Transfer tart to the oven; bake until custard has set and is slightly golden, 45 to 50 minutes. Garnish with remaining raspberries. Let rest at least 25 minutes before cutting. Serve warm or at room temperature.