The buckwheat flour adds a deep assertive flavor to this dessert; the flour is also highly nutritious.
Make topping: In a small bowl, combine raisins and brandy; let stand until raisins are plump, about 20 minutes. Drain raisins; discard liquid.
In a 10-inch seasoned cast-iron skillet, melt butter over medium heat. Sprinkle with brown sugar and cinnamon, and remove from heat. Drizzle with corn syrup. Strew pecans, apple, and raisins over corn syrup; set aside.
Preheat oven to 350 degrees. Make cake: In the bowl of an electric mixer fitted with the paddle attachment, cream butter until smooth. Add sugar; beat until light and fluffy. Add eggs one at a time, beating until combined after each addition.
Into a large bowl, sift together flours, ground ginger, baking powder, and salt. In another bowl, combine milk and fresh ginger. Alternately add the flour and milk mixtures to butter mixture, beating until just combined after each addition, beginning and ending with flour mixture.
Pour batter into prepared skillet; make top even using an offset spatula. Bake until golden brown and a cake tester inserted in center comes out clean, about 50 minutes. Transfer to a wire rack; let stand 5 minutes before inverting onto serving platter. Serve warm or at room temperature.