To be fair

A phrase unbelievably overused in the United Kingdom. Mostly by morons to be fair.

To be fair, I haven't bought the milk.

To be fair, I have no idea how to use the phrase correctly.