Stir together ground coffee and water in a pitcher. Cover and let sit overnight at room temperature.
Strain coffee through a fine sieve lined with a coffee filter, letting it drip into another container. Refrigerate up to 3 days.
To serve, fill tall glasses with ice. Add 1/3 cup cold coffee to each, then top each with milk.
