Preheat broiler with rack about 8 inches from heating element. Make diagonal slashes 1/4 inch deep at 2-inch intervals across flesh side of fish. Stir together garlic, oregano, lemon zest and juice, 1 teaspoon salt, 1/4 teaspoon pepper, and 2 tablespoons oil; spread on both sides of fish, rubbing into slashes. Cut fennel stalks from bulb; reserve 1/4 cup of picked fronds. Place stalks lengthwise on a rimmed baking sheet. Top with fish, slashed side up.
Cut fennel bulb in half lengthwise; remove and discard core. Thinly slice bulb lengthwise and toss with tomatoes, olives, remaining tablespoon oil, 1/2 teaspoon salt, and 1/4 teaspoon pepper; spread evenly around fish. Broil until fish is just cooked and opaque throughout and vegetables are charred in places, 8 to 10 minutes. (If they begin to blacken, tent with foil.) Divide fish and vegetables among 4 plates; garnish with fennel fronds.