To call someone down to 59th street means that they have insulted or wronged you in some way and that you are now making the proposition to fight and/or confront them.
Emerging from a "Team spider Christmas special" where a caller insulted all the members of the band that was preforming, they, in turn, called him down to 59th street. (presumably where they were filming this particular show)
1: "...and look at that bass player what a fucking loser"
2: " Hey why don't you come down to the studio and we'll kick your fucking ass!"
3: "Bitch, 59th street!"
13👍 5👎