Anderson got the ball to bounce and tail slightly away from the left-handed Deonarine, who feathered a catch to wicketkeeper BJ Watling and was dismissed for 52 about 15 minutes before the tea break.
Bravo was 146 not out while Denesh Ramdin was on four as the visitors reached 314 for five at tea, still needing another 86 runs to make the hosts bat again on the fourth day of the match at University Oval in Dunedin.
The pitch was providing little assistance to the bowlers and was still a good wicket for batting, with Deonarine and Bravo's partnership proving frustrating for the home side.
Deonarine had survived a difficult chance when he was 40, hammering the ball to short cover where Brendon McCullum dropped the catch. West Indies were 272 for four at the time.
Bravo had brought up his fifth Test century before the break when New Zealand had hopes of running through the West Indies order when they took two early wickets, one of which was the critical wicket of Shivnarine Chanderpaul.
The 39-year-old, who surpassed 11,000 career Test runs in the first innings on his way to 76, was trapped lbw for one by left arm seamer Neil Wagner, who got some reverse swing with the old ball.
Bravo and Marlon Samuels had resumed on 168 for two but were unable to really get going again as New Zealand maintained a tight line and length before Tim Southee reacted sharply to dismiss the latter caught and bowled.
The right-hander's dismissal for 23 left the visitors at 178 for three and when Chanderpaul was adjudged lbw seven runs later there was a noticeable lift in the New Zealanders' body language as they sensed the possibility of victory.
McCullum had enforced the follow on before tea on the third day when they dismissed West Indies for 213 in their first innings.
New Zealand, who made 609-9 declared, have not won a Test in more than a year, having drawn five and lost four of their matches since McCullum took over the captaincy in late 2012.