Take you for a mug

It's more like saying the joke is on you. Whomever is taking you for a mug, is taking you for an idiot.

Don't let them take you for a mug, you're smarter than that.

19👍 5👎