Englishman Rose bloomed in the heat of the desert, charging through the field with eight birdies and an eagle before being thwarted in his victory attempt by McIlroy's outstanding late run of five closing birdies in as many holes.
"That was a lot of fun," Rose said after finishing on 21-under 267, two strokes behind world number one McIlroy.
"The front nine was just as good as the back nine and on the back nine the putts just started going in. It was important for me to stay positive and aggressive because there were a lot of birdie opportunities out there."
Rose's round bettered the previous mark of 64 set by Sergio Garcia (twice) and his fellow Spaniard Alvaro Quiros, last year's winner here.
Britons Lee Westwood and Ross Fisher, Martin Kaymer of Germany, Swede Peter Hanson and Indian Jeev Milkha Singh have also posted eight-under rounds in this tournament.
Rose said he thought his 64 was going to be enough to earn him the title.
"I thought it was going to be good but I also knew that that's what Rory does," he added.
"If you watched the way Peter Hanson won the BMW Masters in Shanghai last month, Rory pushed him all the way to the end by making late birdies.
"I figured he might do the same here."