Bring milk, sugar, cocoa powder, and chocolate to a simmer in a medium saucepan over medium heat, stirring until chocolate has melted (do not let boil). Pour into mugs.
Grease a heart-shaped petit four cutter or aspic cutter with vegetable oil or cooking spray. Cut several marshmallows in half horizontally, and then press the cutter into the center of each one. Float the marshmallows and the cutouts on hot chocolate.