Preheat the oven to 350 degreesÂ with two racks in center. Line bottoms of three 8-by-2-inch buttered cake pans with parchment. Dust bottoms and sides with flour; tap out excess. Sift flour, baking powder, salt, and ground ginger into a medium bowl. Add ground pecans, and whisk to combine; set aside.
In the bowl of an electric mixer fitted with the paddle attachment, cream butter until light and fluffy. Gradually add brown sugar until fluffy, 4 to 5 minutes. Drizzle in eggs, a little at a time, beating each time until batter is no longer slick, about 5 minutes; scrape down sides twice.
With the mixer set on low, add flour mixture to sugar-butter mixture, alternating with milk, a little of each at a time, starting and ending with flour mixture; scrape down bowl twice. Beat in grated ginger and vanilla. Divide batter evenly among pans. Bake until a cake tester inserted into center of cakes comes out clean, 35 to 40 minutes, rotating pans if necessary for even browning.
Transfer pans to wire racks to cool, about 15 minutes. Remove cakes from pans, and return to wire racks, tops up; let cool.
Meanwhile, place creme fraiche in a medium bowl, and whip with an electric hand mixer until soft peaks form. Add bourbon and confectioners' sugar; beat until soft peaks return. Cover, and refrigerate to firm, about 1 hour.
To assemble cake, remove parchment from bottoms of cakes. Set aside prettiest layer; it will be used for top of cake. Place one layer on serving platter, and spread with half the creme-fraiche filling; repeat with second layer. Top cake with reserved third. Chill cake while icing is being prepared. Pour the icing onto the center point on the top of the cake, and let it flow down the sides. When the icing has set slightly, 5 to 10 minutes, arrange the pecan halves and crystallized ginger on the top. Serve.