Place potatoes in a medium pot; cover with 2 inches water. Season water generously with salt. Bring to a boil, then reduce to a simmer. Cook potatoes until tender, 10 to 12 minutes. Drain. When just cool enough to handle, cut in half lengthwise.
Meanwhile, cook bacon over medium heat until fat is rendered and bacon is crisp, 10 to 12 minutes. Transfer to a paper-towel-lined plate to drain. Break into bite-size pieces.
Transfer potatoes to a large bowl; gently toss with oil and vinegar. Season with salt and pepper; fold in celery and leaves, watercress, and bacon. Serve warm or at room temperature.