Bring a large pot of salted water to a boil. Add sweet potatoes and return to a boil, then reduce heat and simmer 8 minutes. Add cauliflower; return to a boil, then reduce heat and simmer until potatoes and cauliflower are very tender, 10 to 12 minutes. Drain and return vegetables to pot. Place over medium heat and cook, stirring, until a film forms on bottom of pot, 1 to 2 minutes.

Combine milk, butter, nutmeg, and cayenne in a small pot. Bring just to a simmer. Add to cauliflower-potato mixture and mash. Season with salt and pepper. Serve immediately, or keep warm in a heatproof bowl set over a pot of simmering water up to 1 hour. Mash can be rewarmed by adding about 1/4 cup milk and stirring over low heat.