Set a steamer basket in a large saucepan. Fill with enough water to come just below basket. Bring to a boil; place potatoes in basket, and reduce to a simmer. Cover, and steam until tender, 15 to 20 minutes.
Meanwhile, in a large bowl, whisk together oil, mustard, and vinegar. Add celery, scallions, and hot potatoes, and season with salt and pepper; toss to combine. Cool to room temperature, tossing occasionally, about 1 hour.
Toss the potatoes with the dressing while they're hot to help them absorb the flavors.