Using an electric mixer, beat butter, sugar and salt in a large bowl, scraping down sides and bottom of bowl as needed, until light and creamy, about 3 minutes.

Add egg yolks and vanilla and beat until incorporated, about 1 minute.

Reduce speed to low, add 2 cups flour, and mix until incorporated. Dough will be semifirm but can be rolled out without chilling. If your kitchen is very warm and/or dough is sticky, pat it into a 1"-thick disk, wrap in plastic, and chill 15 minutes to let firm up before rolling out.

Preheat oven to 350°.

Roll out dough on a lightly floured sheet of parchment paper, dusting with more flour as needed to keep dough from sticking, to about ¼" thick.

Punch out shapes as desired with lightly floured cookie cutters and transfer to 2 parchment-lined baking sheets, spacing 1½" apart.

Pat scraps into a 1"-thick disk, wrap with plastic, and chill 10 minutes if soft. Repeat process.

Bake cookies 10-15 minutes, rotating halfway through, until edges are golden brown.

Let cookies cool 10 minutes on baking sheet, then transfer to a wire rack and let cool completely. 

Whisk 1 cup powdered sugar, 3 Tbsp. milk, food coloring (start off with just a few drops and add more as needed), and half of salt in a small bowl until no lumps remain. Glaze should be the consistency of heavy cream; add more milk if needed.

For a deeper color, whisk in more food coloring 1 drop at a time until desired color is achieved; adjust with milk if needed. Repeat with second food-coloring, remaining salt, remaining 1 cup powdered sugar, and remaining 3 Tbsp. milk, adjusting intensity of color as desired.

Spoon 1 tsp. first glaze in the center of a small plate. Drizzle 1 tsp. second glaze over the first and use a toothpick to slightly swirl colors together; don’t go overboard. (If you want to decorate larger cookies, add 1 tsp. glaze to plate for each additional inch.)

Gently press the top of a cookie into glaze, then lift up and allow excess glaze to drip back onto the plate.

Using a toothpick, pop any air bubbles and swirl colors more if desired.

Transfer to a wire rack set inside a rimmed baking sheet.

Repeat with 2 more cookies, adding ½ tsp. of each color of glaze to plate before each cookie is dipped.

The glaze will lose its swirly effect at this point. Clean plate and repeat process, decorating cookies in batches of 3 until all cookies have been glazed.

Let sit until glaze is set, at least 2 hours.