Homemade ice cream with an ice cream maker is always a fun pastime for adults and kids alike.
Line one 8-by-12-inch glass or metal baking pan and one 8-by-8-inch glass or metal baking pan with parchment paper.
Fill a large bowl with ice and water; set aside. Combine egg yolks and sugar in the bowl of an electric mixer fitted with a whisk attachment. Beat at medium-high speed until very thick and pale yellow, 3 to 5 minutes. (Alternatively, whisk egg yolks and sugar together in a medium bowl.)
In a medium saucepan, bring milk just to a simmer. With mixer running on low, use a measuring cup or ladle to slowly pour about 1/2 cup of the hot milk into the egg-yolk mixture, beating constantly until blended. Continue, adding about 1/2 cup milk at a time, until all has been added.
Return mixture to saucepan; cook over medium-low heat, stirring constantly with a wooden spoon, until mixture is thick enough to coat back of spoon, 3 to 5 minutes. The custard should retain a line drawn across back of spoon with your finger. Whisk in 1 cup peanut butter.
Remove pan from heat, and stir in cold heavy cream. Pour custard through a mesh strainer into a medium bowl set in the ice bath; stir occasionally until cool. Stir vanilla into cooled custard. Cover bowl and refrigerate at least 30 minutes or overnight.
Pour the custard into a chilled ice-cream maker; freeze according to manufacturer's instructions. (The ice-cream mixture will be set but not hard.)
Swirl in remaining cup of peanut butter; transfer two-thirds of the soft ice cream, or 1 quart, to larger baking pan, smoothing the surface with an offset spatula. Transfer remaining third (1/2 quart) to smaller pan, smoothing its surface. Cover each pan with plastic wrap; freeze overnight. Ice cream may be made to this point up to 1 week ahead.