Preheat oven to 350 degrees. Place potatoes and garlic on a rimmed baking sheet. Drizzle with olive oil and season with salt and pepper. Toss to coat. Roast until dark golden brown, 30 to 45 minutes. During the last 5 minutes of roasting, increase the temperature to 500 degrees and sprinkle with rosemary. Remove from oven, and serve.