Substitute Scott Rendell scored from close range 10 minutes from time to seal a famous win over a Premier League team ranked 85 places above them in the league ladder.
Rendell drifted to the near post and poked the ball into the net to convert JJ O'Donnell's cross to cap off a brilliant performance from the Conference National side.
Luton's famous victory makes them the first non-league outfit since Sutton United in 1989 to beat top flight opposition in the Cup, Sutton having famously beaten 1987 FA Cup championc Coventry City that year.
The Hatters' win against Norwich was arguably better even that that win, however, given that it took place at Carrow Road: the last time a non-league side beat top-flight opposition away from home was Altrincham's victory over Birmingham City at St Andrew's in 1986.
Norwich boss Chris Hughton rested several players, but as Luton - in particular goalkeeper Mark Tyler - held firm, club captain Grant Holt was deployed from the substitutes' bench at half-time.
Holt brought a great save from Tyler with a diving header, and soon after Anthony Pilkington and Wes Hoolahan also came on as Norwich pushed for the win.
But then came Luton's sucker punch - Alex Lawless had just volleyed wide when O'Donnell got free down the left, and Rendell showed great desire to beat the defenders to the ball and complete an FA Cup miracle.
At Loftus Road there was a result every bit as extraordinary as MK Dons destroyed QPR 4-2 on an afternoon that saw two of the most extraordinary FA Cup upsets of recent years.
QPR's atrocious season took an embarrassing turn for the worse as the Dons knocked them out with alarming ease at Loftus Road.
With Tuesday's clash against Manchester City in mind, the Premier League's bottom club made nine changes and went behind after four minutes through Armand Traore's own goal.
QPR dominated the first half but paid for wasteful finishing when Ryan Lowe raced clear to extend the League One side's lead five minutes before the break.
Substitute Ryan Harley netted from close range five minutes after the restart and Darren Potter extended the Dons' lead even further to the disbelief of the 3,155 visiting fans, who even cheered when Jay Bothroyd pulled one back and saw Fabio net in stoppage time.
Another top flight club, Wigan Athletic, survived a scare against non-league Macclesfield Town as they won 1-0, but the visitors were forced to hang on to their slender lead for 83 minutes following Jordi Gomez's early penalty.
Top flight Reading had no such trouble against League One side Sheffield United, with two goals by Noel Hunt and one apiece from Garath McCleary and Mikele Leigertwood wrapping a 4-0 win up before the hour mark.
Everton struck deep in injury time to earn a 2-1 win over Championship side Bolton Wanderers. Steven Pienaar opened the scoring for the Toffees on 18 minutes before Marvin Sordell popped up ten minutes later with a goal that looked like securing a replay for Dougie Freedman's men, only for Johnny Heitinga to find the net in the dying moments.
There was also late drama as League Two Aldershot Town almost enjoyed a superb afternoon as they lost 2-1 to Championship side Middlesbrough. Boro had appeared to pinch it on 83 minutes as Lukas Jutkiewicz struck, but Danny Hylton then seemed to have earned Aldershot a replay with his injury time goal.
Yet there was still time left, and Jutkiewicz scored again to earn Boro passage into the fifth round.
Huddersfield and Leicester City will also meet again after Chris Wood popped up eight minutes from time for the Foxes to cancel out Lee Novak's penalty and secure a 1-1 draw.
Blackburn Rovers cruised into the fifth round with a 3-0 demolition job away at Derby County, with Colin Kazim-Richards, Scott Dann and Jordan Rhodes all on target, while Barnsley also made it through with a 70th minute goal by Chris Dagnall giving them a 1-0 win over Championship high-fliers Hull City.