the ghost/spectre at the feast
(British literary) something or someone that spoils your enjoyment by making you remember something unpleasant John was the spectre at the feast, always reminding her of her broken promise.
raise the spectre of something
(British, American & Australian) also raise the specter of something (American) to make people worry that something unpleasant will happen Drought and war have raised the spectre of food shortages for millions of people. Napoli's 1-0 defeat at Bologna raised the spectre of relegation for the Italian champions.
See the ghost at the feast