Eight-times world billiards champion Advani disposed of 2005 world champion Shaun Murphy in the first round and wasted little time in dispensing with the man who won the Crucible title the following year.
Breaks of 63 and 73 in the first two frames put him in control before Dott pulled one back with an 84.
However a 75 run in the following frame put him within one of victory - and he compiled a break of 100 dead to seal a sensational win.
He will face Judd Trump in the last eight after the world number two beat Andrew Higginson 4-1.
2010 world champion Neil Robertson was a second-round casualty as Stuart Bingham beat him by the odd frame 4-3.
Ding Junhui saw off a brave fightback by Mark Allen to win their tie 4-2.
The Chinese star broke big to race into a 3-0 lead but twice failed to finish the job allowing the Northern Irishman to put together his own impressive runs and pull two frames back.
He left an opening early in frame six, however, allowing Ding to get back on his game and move onto the quarters where either Robert Milkins or Sam Baird await.
Alan McManus booked a quarter final meeting with Stephen Maguire with a superb comeback from 3-0 down to beat Joe Perry 4-3.
Perry took the opener with his second scoring visit and a superb 80 was enough to give him a two frame lead after pouncing on another McManus mistake. Perry edged a tense third punctuated by long bouts of safety to put himself within a frame of the quarters but six straight snooker escape misses gave McManus the impetus to get his first frame on the board.
The Scot won a scrappy fifth to pull a further frame back and made the most of his first visit to the table to take it to a decider. And two heavy scoring visits proved enough to seal a memorable comeback.
And Robert Milkins booked a quarter-final meeting with defending champion Ding Junhui with a gritty 4-2 win over Sam Baird.
Milkins came from 40-0 behind to grab the opener with a 78 break and looked like pinching the second with only the colours required. But a missed black off its spot let a relieved Baird in to level. Milkins put the disappointment of the missed black behind him in the next with an 88 break to edge ahead again.
Baird levelled matters with a composed 88 break but again squandered a healthy 46 point lead as Milkins pinched the next on the pink. Baird again looked in control in the next with Milkins one snooker escape away from needing snookers. But a superb clearance from red to black sealed frame and match by a solitary point.
Round two results:
Ken Doherty 4-3 Tom Ford
Pankaj Advani 4-1 Graeme Dott
Neil Robertson 3-4 Stuart Bingham
Judd Trump 4-1 Andrew Higginson
Stephen Maguire 4-2 Matthew Stevens
Ding Junhui 4-2 Mark Allen
Alan McManus 4-3 Joe Perry
Robert Milkins 4-2 Sam Baird