Toss the herbs into a saucepan, and then cover them with cold water. Bring it to a boil over medium-high heat, and then immediately turn the heat down to low. Simmer, uncovered, for 20 minutes.
Strain the liquid through a fine-mesh sieve, and then discard the spent herbs. To serve right away, pour the coffee into four mugs, or transfer it to a jar and store it in the fridge up to 3 days.
Frothing the milk.
Pour the milk into a milk frother, and then add the vanilla extract. Froth the milk according to your machine's directions. If you don't have a frother, you can pour the milk into a small saucepan and then whisk it vigorously over medium-low heat until warm.
Pour the milk into the mugs of coffee, and serve hot.