drift apart

(from each other )

1. . Lit. [for floating things] to separate as they drift. The boats drifted apart from one another. The boats drifted apart in the waves.

2. Fig. [for people] to lead their lives without contact with each other having been together or friendly. He drifted apart from his friends. As the years went by, they drifted apart.