Preheat oven to 400 degrees with rack in middle position. Spread butter across bottom and sides of a 2-quart baking dish.
Peel potatoes and slice 1/8 inch thick. Place slices in a large bowl of cold water and drain thoroughly.
Bring heavy cream, milk, and garlic to a simmer in a large, wide saucepan. Add potatoes, salt, and pepper and continue to simmer, stirring occasionally, until potatoes are tender, 20 minutes.
Using a slotted spoon, transfer half the potatoes to prepared baking dish and sprinkle with half of the cheese. Add remaining potatoes to baking dish and pour enough cream mixture over potatoes to just cover. Sprinkle with remaining cheese, place on a rimmed baking sheet, and bake until bubbly and golden brown on top, 40 to 45 minutes.
Remove from oven and grate nutmeg over top just before serving.