Place potatoes in a large saucepan, and cover with cold water. Bring to a boil, and simmer until just tender but still quite firm when pierced with the tip of a sharp knife, 15 to 20 minutes. Remove all but 4 potatoes; cook these until soft, about 5 minutes.
When cool enough to handle, carefully cut off the very tops of potatoes removed in Step 1. Using a melon baller, scoop out as much flesh as possible while leaving at least a 1/4-inch-thick wall remaining. Place flesh in a small metal bowl set over a pan of simmering water. Peel and add remaining 4 potatoes. Cut a small slice off the bottom of each potato shell so it will stand upright.
Preheat oven to 375 degrees. Heat milk and butter in a small saucepan until butter melts. Push potato flesh through a ricer, food mill, or wide-mesh sieve into a bowl. Stir in milk and butter a little at a time until creamy. Stir in crème fraîche, salt and pepper to taste, and 1 tablespoon chives.
Place potato mixture in a pastry bag fitted with a large plain tip. Arrange potato shells on a baking sheet. Pipe filling into shells, making sure to cover cut edges.
Bake for 15 minutes, or until golden brown and slightly puffed. Brush with melted butter, and bake for 15 minutes more. Top each potato with a spoonful of caviar, and sprinkle with chives.