leave a sour taste in (one's) mouth

To have a persistent negative effect on someone after an offending incident has ended. I know Kelly's your friend, but she was so rude at dinner—it just left a sour taste in my mouth. Of course Andrew not inviting me to his party leaves a sour taste in my mouth.