Connacht were outscored two tries to one but Kieran Marmion's second-half effort combined with 11 points from the boot of fly-half Dan Parks - who successfully kicked two penalties, a conversion and a drop goal - saw them claim a huge scalp.
Four-time champions Toulouse notched tries in either half through Jean-Pascal Barraque and Thierry Dusautoir but they could not find a way to break down the Connacht defence again in the closing stages as they slumped to a rare home defeat in Europe.
And the French giants could even have lost by more points in the end had the TMO not spotted a knock-on after Robbie Henshaw had streaked away for what seemed like a brilliant second-half try for the visitors.
Connacht had turned desperate defence into thrilling attack after Toulouse had turned the ball over in the visitors' 22. Gavin Duffy and Marmion had combined to send the full-back racing clear, but Romanian referee Marius Mitrea asked the TMO to check the origin of the move.
Unfortunately for Connacht the TMO discovered a slight knock-on some 70 metres back in the play by a Connacht player and they missed out on seven points.
That could have been critical because seven minutes later Toulouse kicked to the left corner and rumbled skipper Duautoir over for a try which Lionel Beauxis converted of the touchline.
That cut the gap to two points and set-up a thrilling final 15 minutes in which Connacht never stopped taking the game to their illustrious opponents as well as producing superb defence.
Parks missed with a drop-goal and a 50-metre penalty in those closing minute to add to the drama, but there was no denying the Irishmen.
Following on from Leinster's win at Northampton, Ulster's triumph over Benetton Treviso and Munster's bonus point victory against Perpignan, Pat Lam's Connacht side had the whole of Ireland urging them on to make it a clean sweep.
Toulouse had not been beaten at Stade Ernest Wallon for five years in the tournament, yet they only led for five minutes in the match. Parks put his side ahead with a 13th-minute penalty and then added a drop-goal three minutes later to make it 6-0.
It did not help the home cause that fly-half Barraque missed with two penalty kicks, but he did make up for his first mistake with a brilliant individual try on the stroke of half- time.
He dummied past Craig Clarke, weaved past Henshaw and touched down at the posts for a try which he converted to make it 7-6 to the home side at the break.
That lead did not last for long though, as Parks hammered over a penalty five minutes into the second half and then scrum-half Marmion rounded off a brilliant right touchline move involving Henshaw and Fionn Carr by burrowing over for a try which was finally confirmed by the TMO.
Parks converted superbly off the touchline and that gave Connacht a 16-7 lead with just over half-an-hour to play. They then defended superbly to take the four points and put themselves right back into the reckoning for a quarter-final place.
Despite their defeat Toulouse still stay on top of the pool with 10 points, but Saracens are level on points with Connacht now one point behind with a return game against the French side in Galway to come next weekend.