The tie finished 2-2 after 90 minutes but Brian Howard, Liam Walker, Ashley Harris and Luke Rodgers all converted penalties for Pompey, while Lee Barnard saw his effort saved by Mikkel Andersen and Marc Pugh blazed over to send the Cherries crashing out.
Bournemouth took just five minutes to take charge as Shaun MacDonald nodded home Simon Francis cross from the right.
However Pompey drew level just before first-half stoppage time, Mustapha Dumbuya's persistence down the right paid off and Luke Rodgers slid in to convert his cross.
The visitors edged ahead again courtesy of MacDonald's second of the match, his side-footed effort from the edge of the box whistling straight into his top corner.
Wes Fodgen was inches from extending Cherries; lead after 66 minutes when he turned on to Harry Arter's shot and fired it against the foot of a post. And the home side hit back immediately as skipper Brian Howard blasted them level from 25 yards.
Matt Smith was the hero and villain for Oldham as Chesterfield began the defence with a 2-1 victory over the team they beat in last season's Northern Area final. Smith gave Oldham a first-half lead but scored an own goal in added time after Danny Whitaker had equalised and then saw a penalty saved by Dean Bouzanis.
Coventry's first appearance ended in a 10-9 penalty shoot-out victory following a 0-0 draw with Burton Albion.
Preston best Carlisle 3-1 on penalties after the sides had fought out a 1-1 draw in normal time in the first-round tie at Brunton Park. David Buchanan fired home the decisive spot-kick after the home side had missed three times in succession. Chris Beardsley had given Preston a third-minute lead which was cancelled out in the second half by Jon-Paul McGovern to send the tie to penalties.
Jennison Myrie-Williams netted a second-half double to send Port Vale into the second round with a comfortable 2-0 victory against disappointing Tranmere.
Notts County came from behind to win 2-1 at Scunthorpe - their first win in the competition for 11 years.
Dagenham & Redbridge sprung a surprise by dumping League One Stevenage out of the at the first hurdle courtesy of a fine 3-2 win.
Ryan Cresswell's last-gasp header handed Southend a 2-1 win over AFC Wimbledon in the first round at Roots Hall.
Jake Robinson's wonder goal was enough for Northampton to see off MK Dons 1-0.
Jack Redshaw and Kevin Ellison took Morecambe through to the second round of with a 2-0 win over Lancashire rivals Accrington.
A superb 25-yard strike from substitute Scott Neilson sent Crawley into the second round with a 3-2 home win over League Two leaders Gillingham.
Rochdale won a penalty shootout 4-2 to edge past Fleetwood in the first round. The game ended 2-2 after 90 minutes with the visitors coming back to cancel out Dale's 2-0 lead.
Two quality free-kicks from Ed Upson helped Yeovil to a 3-0 first-round win over West Country rivals Bristol Rovers at the Memorial Stadium.
Brave York withstood almost constant Rotherham pressure to grab a 1-0 victory with a late winner from Matty Blair.
Aldershot moved into the second round after winning 4-3 on penalties at Exeter, with the tie having finished goalless.