Using a paring knife, lightly score an X into bottom of peaches (this will help with peeling later). In a large saucepan, combine sugar, vanilla, lemon zest, mint, and water. Cook over medium, stirring occasionally, until sugar has dissolved, about 2 minutes.
Add peaches and enough water to cover. Bring to a bare simmer, and cook, turning occasionally, until peaches are easily pierced with a skewer, 6 to 10 minutes, depending on ripeness of peaches.
Remove peaches with a slotted spoon; let cool slightly. Using a paring knife, peel peaches; return to syrup. Serve with mint sprigs.