I think this phrase actually began in basketball. If there isn't any harm inflicted to stop a game, then there should be no foul called.
It's used commonly today to brush off insults or mistakes and the like.
Gerchewd: Hey sorry - my butt just accidentally grazed your arm.
Doughlas:Hey man, no harm no foul.
Gerchewd: Yyyyyeah.
55👍 44👎