Skipper Lee Carey's second-half penalty looked like being enough for Hastings as they led until the final minute of normal time.
But Tom Platt, who scored Harrogate's goal in the 1-1 draw the first time the teams met, was on hand at the back post to take the game into extra-time before Okojie's heroics in the shoot-out.
The Isthmian Premier side - the lowest-ranked team left in the competition - looked overwhelmed at the Pilot Field in the first half and were fortunate to still be level at the break after Luke Dean and Platt both went close.
However, just two minutes after half-time they took the lead when Zac Attwood was fould by Matt Bloomer in the box and Carey's placed effort put them in front from the spot.
Platt's equaliser came in the 90th minute as Harrogate looked to have run out of ideas, making the most of Matt Armstrong-Ford's failure to deal with a corner to poke home.
After a cagey period of extra-time Platt missed Harrogate's second penalty, blasting over the bar, but Danny Ellis did the same when given the chance to win it for the Arrows.
Lee Elam then proceeded to fire over with the first penalty of sudden death and Okojie coolly found the right-hand corner to seal a place in history for the hosts.
Revsed third round draw:
Crystal Palace v Stoke City
Brighton and Hove Albion v Newcastle United
Tottenham Hotspur v Coventry City
Wigan Athletic v Bournemouth
Fulham v Blackpool
Aston Villa v Ipswich Town
Charlton Athletic v Huddersfield Town
Barrow or Macclesfield Town v Cardiff City
Barnsley v Burnley
Manchester City v Watford
Swansea City v Arsenal
Leicester City v Burton Albion
Millwall v Preston North End
Cheltenham Town v Everton
Derby County v Tranmere Rovers
Crawley Town v Reading
Aldershot v Rotherham United or Notts County
Middlesbrough v Hastings United
Accrington Stanley or Oxford United v Sheffield United
Southampton v Chelsea
Queens Park Rangers v West Bromwich Albion
Peterborough United v Norwich City
Mansfield Town v Liverpool
Bolton Wanderers v Sunderland
Nottingham Forest v Oldham Athletic
West Ham United v Manchester United
Hull City v Leyton Orient
Blackburn Rovers v Bristol City
Leeds United v Birmingham City
Southend United v Bradford City or Brentford
Luton Town v Wolverhampton Wanderers
Sheffield Wednesday v MK Dons
