Put cream cheese into the bowl of an electric mixer fitted with the paddle attachment; mix on medium speed until smooth. Set aside. Whisk creme fraiche in another medium bowl until the whisk begins to leave lines, about 1 minute. Whisk 1/3 of creme fraiche into cream cheese. Add remaining creme fraiche, beating until smooth and scraping down sides of bowl as needed. Sift confectioners' sugar over top, and fold to combine. Stir in vanilla. Refrigerate at least 1 hour (up to 5 hours).