Preheat grill to medium-high. (If you are using a charcoal grill, coals are ready when you can hold your hand 5 inches above grill for just 3 to 4 seconds.) Place lamb chops on a plate, and season both sides with salt and pepper. Drizzle lightly with oil, and turn to coat.
Place potatoes in a medium saucepan, and add enough cold water to cover potatoes by about 2 inches. Add 1 1/2 teaspoons salt, and bring to a boil. Reduce heat to medium-low, and simmer until potatoes are just tender, 15 to 20 minutes. Drain, and let cool slightly. Halve potatoes, and toss with 1 tablespoon oil; season with salt and pepper.
Toss bell peppers with 1 teaspoon oil. Place bell peppers and potatoes on grill. Cook peppers, turning often, until tender and blackened in places, 10 to 12 minutes. Cook potatoes, cut side down, turning after bottoms are golden brown and no longer stick to grill, 8 to 10 minutes total. Transfer vegetables to a platter, and cover with parchment paper and then foil to keep warm.
Place lamb chops on grill. Cook, flipping once, until an instant-read thermometer registers 130 degrees for medium-rare, 4 to 5 minutes per side. Transfer to platter with vegetables, and let rest.
Combine garlic, chopped anchovies, and remaining 2 tablespoons oil in a skillet, and place on grill (or over medium heat on cooktop). Cook, stirring, until garlic softens and anchovies sizzle and break down, about 4 minutes. Stir in parsley and lemon juice.
Arrange lamb, whole anchovy fillets, and vegetables on plates. Drizzle with anchovy-parsley sauce, and garnish with parsley. Serve immediately.