Residual semen that's been left on an article of clothing, therefore dried and crusty. The 'vanilla' come from the color of the jizz which resembles a sort of melted vanilla ice cream.
Bob: Look at that girl! She's got some vanilla spots on her pants.
George: Walk of shame maybe?