Substitute Danny Crow cancelled out Ashley Barnes' opener before scoring again in the first period of extra-time along with fellow sub Conor Washington.
The Seagulls broke through to take the lead after 18 minutes when unselfish play from Will Buckley set up striker Barnes and he coolly rolled the ball into the net from close in.
The hosts were reduced to 10 men 23 minutes from time when Spanish defender Inigo Calderon was shown a straight red card by referee Brendan Malone following a challenge on Byron Anthony.
Newport capitalised on their numerical advantage to pull level nine minutes from time through substitute Crow, who smartly headed in a cross from Adam Chapman.
Striker Crow struck again to put Newport ahead in the fourth minute of extra time with a smart finish into the top corner past Casper Ankergren before substitute Washington pounced on a defensive mistake to make it 3-1 in the 102nd minute.
Middlesbrough were also shocked as they were beaten 2-1 by Accrington.
Lukas Jutkiewicz had put the hosts ahead but Marcus Carver and Piero Mingoia hit back to give Stanley a famous win.
Ipswich were another championship side to be knocked out by lower-league opposition as they fell to a 2-0 defeat at Stevenage.
This is the third season in a row that the Tractor Boys have been knocked out of this competition by lower-league opposition as two second-half goals sent the League One outfit through.
A penalty from Filipe Morais early in the second half put Stevenage ahead, with Jordan Burrow sealing the win with 15 minutes left.
League One Rotherham upset local rivals Sheffield Wednesday with a 2-1 win.
Ben Pringle put the Millers ahead before Rhys McCable levelled for the visitors.
Lee Frecklington put the hosts back in front before Wednesday substitute Jermaine Johnson was sent off for a headbutt on Pringle.
Last season's beaten finalists Bradford made an early exit as they suffered a 2-1 defeat at Huddersfield.
James Vaughan, a £1million summer buy from Norwich, grabbed a brace before a Nahki Wells free-kick in the last minute set up a tense finish.
Results in full
AET Barnsley * 0 - 0 Scunthorpe
AET Birmingham 3 - 2 Plymouth
FT Bournemouth 1 - 0 Portsmouth
FT Brentford 3 - 2 Dag and Red
AET Brighton 1 - 3 Newport
FT Bristol Rovers 1 - 3 Watford
FT Bury 3 - 2 Crewe
FT Charlton 4 - 0 Oxford
AET Cheltenham 4 - 3 Crawley
FT Colchester 1 - 5 Peterborough
FT Doncaster 1 - 0 Rochdale
FT Exeter 0 - 2 Queens Park Rangers
FT Gillingham 0 - 2 Bristol City
FT Huddersfield 2 - 1 Bradford
FT Leyton Orient 3 - 2 Coventry
FT Middlesbrough 1 - 2 Accrington
FT Millwall 2 - 1 AFC Wimbledon
FT Morecambe 1 - 0 Wolverhampton Wanderers
FT Northampton 1 - 2 Milton Keynes Dons
FT Nottingham Forest 3 - 1 Hartlepool
FT Oldham 0 - 1 Derby
FT Port Vale 1 - 2 Walsall
FT Sheffield United 1 - 2 Burton
FT Shrewsbury 1 - 3 Bolton
FT Southend 0 - 1 Yeovil
FT Stevenage 2 - 0 Ipswich
FT Swindon 1 - 0 Torquay
FT Tranmere 2 - 0 Mansfield
FT Wycombe 1 - 2 Leicester
FT York 0 - 4 Burnley
FT Rotherham 2 - 1 Sheffield Wednesday