Preheat oven to 350 degrees. Coat an 8-inch square or round cake pan with cooking spray. Line bottom with parchment paper, and set aside.
Pulse saffron and 1/4 cup sugar in a spice grinder until well combined. Put butter and saffron sugar into the bowl of an electric mixer fitted with the paddle attachment. Mix on medium-high speed until pale and fluffy, about 3 minutes. Using a small offset spatula, spread mixture into prepared pan.
Peel pears; halve lengthwise, and core. Using a mandoline or a very sharp knife, cut pears lengthwise into 1/4-inch-thick slices. Arrange pears decoratively in pan over saffron-butter mixture.
Whisk together flour, remaining 3/4 cup sugar, the baking powder, ground ginger, and salt in a large bowl. Whisk buttermilk, oil, eggs, and vanilla in another bowl. Add buttermilk mixture to flour mixture in a slow, steady stream, whisking. Stir in crystallized ginger.
Spread batter over pears. Bake until set and a cake tester inserted into center comes out clean, 40 to 45 minutes. Let cool 5 minutes. Place a serving platter upside down over pan; flip to unmold cake. Peel off parchment.