Steep earl grey tea bags in boiling water for 6 minutes.
Add sweetened condensed milk and chill for 30 minutes in the refrigerator.
Add heavy whipping cream (and vanilla bean paste if using) to a large bowl and beat with an electric mixer on medium-high until thickened. You don’t want to make whipped cream, you just want to get a thicker consistency and incorporate some air so the pops are less icy.
Add the tea/condensed milk mixture to the whipped cream and gently mix it together. Add the mixture to a spouted bowl or large liquid measuring cup.
Pour enough into each popsicle mold to fill. Insert popsicle sticks, secure cover, and place in the freezer. Set a timer for 30 minutes and then straighten out the sticks. Continue to freeze for at least 6 hours or overnight.
To remove popsicles from the mold, run each individual popsicle under lukewarm water and gently tug on the popsicles stick until it releases. You can store popsicles in the plastic mold until ready to enjoy.