in the wilderness

someone, especially a politician, who is in the wilderness, does not now have the power or influence that they had before He spent several years in the political wilderness after conservatives objected to his attempts to reform the police.