Preheat oven to 350 degrees. Generously butter a 9-inch springform pan; set aside.
Sift all-purpose flour, baking powder, and 3/4 teaspoon salt together into a medium bowl; stir in semolina flour. Set aside.
Whisk 1 of the egg whites vigorously in a small bowl until foamy. Transfer 2 teaspoons of egg white to a medium bowl; discard remainder. Stir almonds, brown sugar, and a pinch of salt into egg white. Set almond mixture aside.
Put butter, 1 cup granulated sugar, and the almond paste in the bowl of an electric mixer fitted with the paddle attachment. Mix on medium-high speed until pale and fluffy, 3 to 4 minutes. Add egg yolks and vanilla; mix until well blended, about 2 minutes.
Reduce speed to low. Mix in reserved flour mixture in 3 batches, alternating with 2 batches of the milk. Continue mixing until no traces of flour or milk remain, about 2 minutes. Transfer the cake batter to a large bowl.
Fit the electric mixer with the whisk attachment. Put remaining 4 egg whites in a clean mixing bowl; beat on medium high speed until foamy. Gradually beat in remaining 1/4 cup granulated sugar; beat until soft peaks form.
Using a rubber spatula, fold half of egg-white mixture into batter. Gently fold in remaining egg-white mixture.
Transfer the batter to buttered pan. Smooth top with spatula. Bake until cake begins to turn golden but center is still wobbly, 15 to 20 minutes.
Remove cake from oven; scatter reserved almond mixture evenly over top.
Return cake to oven; bake until topping is golden brown and a cake tester inserted into center comes out clean, about 25 minutes. (Check cake 10 to 15 minutes beforehand; if almond topping is darkening too quickly, cover loosely with foil, and continue to bake.)
Let cake cool completely in pan on a wire rack. Run a thin knife around edges of cake to loosen; remove sides of pan. Using two spatulas, transfer cake to a serving platter or a cake stand. Cake can be stored in an airtight container at room temperature up to 4 days.
Use a long serrated knife in a gentle sawing motion to cut through the topping and slice the cake.