drift apart

If two people drift apart, they gradually become less friendly and their relationship ends.

I know that you and I have kinda drifted apart, but you're the only person I knew who lived here in the city.

I drift apart from my friends.