Rovers bounced back from their Hull derby home defeat in spectacular style, producing a tremendous defensive stint to keep the Rhinos at bay in a tense and thrilling finish.
Craig Sandercock's men remain eighth but need just one more win from their last two matches to guarantee their play-off spot, while Leeds missed the chance to consolidate a top-four finish after losing for the first time in five matches.
Leeds captain Kevin Sinfield went through the 3,000 Super League points barrier but the Rhinos will be disappointed they could not pay a fitting tribute to Challenge Cup final winning coach Graham Murray, who passed away last month and to whom the match was dedicated.
There was little hint of what was to come when Leeds' promising loose forward Liam Sutcliffe burst through the attempted tackle of Rhys Lovegrove and Michael Dobson to get centre Kallum Watkins over for the game's first try on 19 minutes.
Sinfield's conversion brought up his 3,000th point and put his side into a 6-0 lead but served only to inspire the visitors, who like their hosts were lacking six regulars through injury.
They drew level after 25 minutes when second rower Jordan Cox regained possession from Dobson's high kick to touch down, although slow-motion replays indicated a possible knock-on by Rovers centre Sean Gleeson during the first point of contact.
The try breathed fresh confidence into Hull KR and Dobson, who was beginning to dictate matters, took hooker Lincoln Withers' pass to race into space and get Lovegrove charging down the wing.
The big Australian still had full-back Ben Jones-Bishop in front of him but he proved unstoppable from 15 metres out and Dobson's second conversion put his side into a 12-6 interval lead.
Dobson increased his side's advantage with a penalty three minutes into the second half as Leeds centre Zak Hardaker was caught offside after team-mate Ryan Hall had knocked on from Dobson's high kick.
Staring at defeat, the Rhinos produced their best spell of the game and laid siege to the Hull KR line but the visitors held firm, although they were relieved when Watkins failed to gather the ball from Sinfield's high kick.
Leeds' error rate grew as they began to chase the game and Dobson kicked a second penalty after 58 minutes to make it 16-6 after Sinfield put a goal-line drop-out straight into touch.
Sinfield almost made amends when his grubber kick eluded Hull KR winger Omari Caro for Hall to get the ball down but the try was disallowed by video referee Ian Smith, who ruled a knock-on.
Rovers also lived dangerously when Hall collected Sinfield's crossfield kick to get Hardaker striding for the line, only to be denied by Travis Burns' last-ditch tackle.
The pressure finally told eight minutes from the end when Hall was halted on the left wing and Sinfield kicked to the other side, where right winger James Duckworth, on his home debut, calmly gathered to get Watkins over for his second try.
Sinfield's second goal cut the gap to four points and Leeds almost scored again straight away when Hall burst down his wing but Sinfield failed to take the final pass, which was a difficult one, with the line open.