to start from the very beginning; to start from nothing. Whenever I bake a cake, I start from scratch. I never use a cake mix in a box. I built every bit of my own house. I started from scratch and did everything with my own hands.
Start at the beginning with no advantage. The scratch line was a stripe across the ground where a race began. Starting from scratch meant having no advantage against others in the race where handicaps allowed some entrants shorter distances to run.