In a large (6-quart) heavy bottomed saucepan, bring milk to a boil over medium-high heat. Add sugar and cook, stirring to dissolve.
Add vanilla beans to saucepan and bring to a simmer. In a small bowl whisk together baking soda and 1 1/2 tablespoons water until baking soda is dissolved. Reduce heat to low and add baking soda mixture to saucepan and cook, stirring constantly. If mixture begins to foam up too high, remove from heat for a moment until foam goes down.
Cook, stirring every 15 minutes or so, until mixture turns brown and is thickened and reduced to 3 cups, 6 to 8 hours. Strain through a fine-mesh sieve and let cool slightly.
Store in an airtight container, refrigerated, for up to 1 month.