Peel celeriac with a paring knife. Cut into 1-inch-thick slices, then cut into very thin ribbons (use a mandoline if you have one), placing in cold water to prevent browning. Drain and blanch briefly in boiling salted water. Squeeze out water, and set aside.
Peel celery and slice on an angle as thinly as possible. Set aside.
In a blender or food processor, combine buttermilk, oil, lemon juice, and salt and pepper to taste. Process until smooth. Add capers and herbs, and process until coarsely chopped, about 5 seconds.
Combine celeriac and celery in a bowl, and toss with dressing.