against (one's) better judgment

In spite of one's apprehension or objections. Against his better judgment, Joe let his daughter attend her friend's party. I allowed my obnoxious co-worker to accompany me on my work trip, against my better judgment.