Place a fine-mesh sieve over a medium bowl; set aside. In a medium saucepan, off heat, whisk together sugar, cornstarch, and salt. Gradually whisk in milk, mixing until ingredients are dissolved. Whisk in egg yolks.
Whisking constantly, cook over medium heat until the first large bubble sputters. Reduce heat to low; continue to whisk, and cook 1 minute. Remove from heat, and immediately pour through sieve into bowl; stir in vanilla.
Divide mixture among ten 3-ounce ice-pop molds. Chill in refrigerator until cool and thickened, about 1 hour. Insert pop sticks, and freeze until solid, at least 4 hours and up to 2 weeks. Run molds briefly under warm water to release pops.