On a miserable, rain-sodden day in northern France, the Briton appeared in great pain after falling for the second time, some 70 kilometres from the finish.
The Team Sky rider stood holding the wrist he injured in another spill on Tuesday, and was limping heavily as he eventually climbed into a team vehicle after chatting to team doctor Alan Farrell and sports director Nicolas Portal.
Earlier in the stage, a 152.5-km ride from Ypres, Belgium, Froome suffered his second crash in two days.
Froome hurt his wrist and suffered bruises to his left side in a fall on stage four but was cleared to start Wednesday's stage which features several treacherous cobbled sectors.
The earlier fall on Wednesday was not too bad and he was helped back into the main bunch by his Sky team-mates, but the second accident of the day proved one too many.
The 29-year-old's build-up to the race was hardly ideal after a heavy crash last month on the Criterium du Dauphine.
Froome is the second high-profile rider to quit this year's Tour after fellow Briton Mark Cavendish suffered a high-speed crash in Saturday's opening stage bunch sprint in Harrogate. Cavendish, winner of 25 Tour stages, underwent shoulder surgery on Wednesday as a result of the crash.
Organisers announced before the start that two of the nine cobbled sectors had been removed from the stage because of the bad weather conditions but crashes still littered the route.
Amongst all the chaos, yellow jersey holder Vincenzo Nibali powered away Spain's Alberto Contador on the cobbled sectors, his main rival for the overall title. With mud plastering his face, Nibali looked light on the cobbles while Contador struggled.
Other riders to crash were triple Paris-Roubaix champion Fabian Cancellara as well as overall contenders Tejay van Garderen, Andrew Talansky, Alejandro Valverde and Jurgen van den Broeck.
See how Froome dealt with his first crash below:
- Sports & Recreation