Gateshead were the better side for much of the first-round replay with Colin Larkin seeing a goal ruled out for offside and James Marwood volleying against the woodwork.
However, Smalley secured Oxford's progress when he rolled in from the spot after James Curtis fouled Ryan Williams in the box with four minutes of extra-time remaining.
As with the original tie, which finished 2-2, Gateshead showed no fear and centre-half Curtis' towering header from Larkin's corner forced a low save from Ryan Clarke in the 20th minute.
Marwood fired wide after half-time from Larkin's through ball while a rare Oxford chance - a Dave Kitson header - was cleared off the line by Jamie Chandler three minutes later.
Larkin thought he had won it for Gateshead in the 100th minute, but his close-range effort was ruled out for offside.
Gateshead continued to press and Marwood crashed a stunning volley against the woodwork but all their work was undone at the death.
Williams swung at a bouncing ball in the box and though he missed his kick he was brought down by Curtis, with Smalley making no mistake.