The result sees Croatia stay second in Group A after Belgium won 2-0 in Macedonia. Both sides have 13 points from five matches with five remaining, Belgium leading by virtue of a superior goal difference.
In the group's other match, Wales won 2-1 in Scotland to move third and put keep the winless Scots bottom.
Serbia have four points from five matches and face a tough task to qualify for next year's finals.
"We knew this game was Serbia's last chance to haul themselves back into the race for one of the top two spots in the group and that's why we had to be patient," Croatia coach Igor Stimac said.
"We are overjoyed and I can only congratulate my players and the staff for doing an outstanding job but this result is history as of now, because we have a tough visit to Wales on Tuesday and anything less than a win would devalue this success."
The match at Dinamo Zagreb's Maksimir stadium was played under tight security amid fears of crowd trouble but apart from sporadic volleys of abuse that pockets of home fans directed at Serbia's players, it was an incident-free event.
Both federations had agreed not to take away fans to either fixture with the reverse fixture set for September 6 in Belgrade.
While Stimac fielded an adventurous 4-4-2 formation, Serbia counterpart Sinisa Mihajlovic surprisingly left out several regular starters and his experiment backfired as the visitors were punished for a cagey approach.
Croatia took the first meeting between the two sides as independent nations by the scruff of the neck after a cautious opening and the Serbians were lucky not to have trailed by a bigger margin at half-time.
Mandzukic fired Croatia ahead midway through the first half when Olic capitalised on a calamitous error by Serbia's left-back Aleksandar Kolarov and squared the ball back to his strike partner, who prodded it home past two defenders from close range.
The 33-year-old Olic doubled the home team's advantage in the 37th minute, chesting in from the edge of the six-yard box a teasing Darijo Srna free-kick after Kolarov was booked for a rash tackle on Mandzukic.
Serbia missed a good chance to pull one back when Croatia keeper Stipe Pletikosa parried Kolarov's stinging shot from 20 yards and Zoran Tosic side-footed the rebound wide with the goal at his mercy.
Croatia then twice went close as Luka Modric's shot from 25 yards sailed just wide and Ivan Rakitic's low cross flashed across the face of goal.
Substitute Filip Djordjevic wasted a golden opportunity for Serbia in the closing stages, directing his shot from 10 yards straight at Pletikosa.
Mihajlovic took the blame for a tepid Serbian performance.
"We conceded two very cheap goals and it was very difficult to come back into the match after that, although we improved in the second half and created some good chances," he said.
"We lost to a better team and I take full responsibility for the defeat because the lads played as well as they could and this is the best they can do at the moment."
Belgium stayed on course for a first World Cup finals appearance in more than a decade after a nervous 2-0 win over a stubborn Macedonia in Skopje.
A neat first-half strike by in-form Kevin De Bruyne and an Eden Hazard penalty just past the hour was the difference between the sides before Belgium host Macedonia on Tuesday.
Belgium, playing without injured captain Vincent Kompany, narrowly averted a disastrous start, with Macedonia striker Adis Jahovik's header only kept out by the back of visiting keeper Thibaut Courtois's legs inside the first minute.
Jahovik had another good chance two minutes later before the highly-rated Belgians, packed with English Premier League talent, settled down to the task at hand.
De Bruyne steadied Belgium's nerves in the 26th minute, dispossessing defender Aleksandar Lazevski and then feigning a pass before placing the ball into the far corner of the net.
Buoyed by their lead, the visitors were more comfortable in the second half and doubled their advantage in the 62nd minute. Hazard was brought down after playing a one-two with Marouane Fellaini and stepped up to score from the spot.
Belgium coach Marc Wilmots was disappointed his defence left too much space and his midfield gave the ball away too often in the first half, telling reporters: "I was angry at halftime".