It's a way of saying that something specific is your weakness. I.E. Stating that you can't help but love it, or that it's your biggest addiction.
Puppy videos are my kryptonite, I just can't get enough.