Two goals in as many second-half minutes from Kayode Odejayi and Lee Frecklington gave the Millers a crucial victory.
Substitute Odejayi coolly picked his spot from 10 yards after Steve Mildenhall had parried Alex Revell's 60th-minute drive.
And before the home side could recover former Bristol City player Odejayi set up Frecklington to smash a volley into the roof of the net from 12 yards.
Joe Anyinsah replied for Rovers with a swerving right-footed shot from the corner of the box with nine minutes left.
A scrappy first half had seen Frecklington waste the best chance when lobbing Mildenhall and clearing the crossbar.
John-Joe O'Toole shot wide for Rovers and Tom Parkes headed over from a corner, but the visitors edged the opening 45 minutes.
After the Rotherham goals Rovers sub David Clarkson forced a fingertip save from Andy Warrington.
AFC Wimbledon failed to make it off the bottom of the table as they were held 1-1 by Northampton.
Alan Bennett opened the scoring for Wimbledon after 28 minutes, but Northampton's Adebayo Akinfenwa scored from the spot with 14 minutes remaining to clinch a point and see his side rise to sixth.
Troy Brown scored the only goal of the game seven minutes from time as Aldershot Town beat Torquay 1-0 to boost their hopes of beating relegation.
