Graham Cummins slotted home the decisive penalty to book a trip to Coventry after Gregg Wylde missed for Bury.
Preston took the lead when Joel Byrom picked out Jack King from a corner and his powerful shot squeezed past Trevor Carson.
Both David Healy and Steven Schumacher missed good chances to equalise, while a threatening Stuart Beavon shot whizzed over the bar at the opposite end.
But Bury eventually levelled when Joe Skarz flicked a well-placed header past Thorsten Stuckmann from Schumacher's free-kick.
An error from Carson allowed Andy Procter to rifle Preston's second but Matt Doherty smashed home Bury's equaliser from a tight angle.
Shane Cansdell-Sherriff was sent off for a second yellow card before Beavon capitalised on another Carson error to force the ball home in the final minute.
Schumacher created further drama with a stunning 25-yard free kick in injury time but Preston crept through after a tense shoot-out.
