From “Leave It To Beaver”, this expression basically means pulling someone’s chain with deliberately provocative banter.
“Oh, you shouldn’t let that jerk Eddie Haskell get to you. He was just trying to give them the business. “