The German driver, who won the season opening race in Melbourne a fortnight ago, recorded a best lap of one minute 39.008 seconds at the Sepang Circuit on Saturday, just ahead of his team mate Lewis Hamilton.
Hamilton had recorded the best time in Friday's first session before being bettered by Rosberg in the second with the Briton struggling for grip in both 90 minute runs and sliding off the hot track.
The 2008 world champion ran wide again in the last 60 minute run and also required his seat adjusting midway through Saturday's session as he finished 0.232 seconds behind the German.
Nico Hulkenberg was fifth for Force India ahead of Vettel's Red Bull team mate Daniel Ricciardo, who was disqualified from the Melbourne race over a fuel flow issue that has continued to dog the team in Malaysia.
Ferrari's Fernando Alonso was down in seventh, suggesting the Spaniard will find it tough to keep pace with the Mercedes during Sunday's race.
Lotus had their most productive practice session in Malaysia with both Romain Grosjean and Pastor Maldonado able to record lap times but the French driver complained that the rear end of the car was 'a disaster' upon driving back to the garage.
McLaren also suffered problems with Danish rookie Kevin Magnussen complaining about a lack of power as he finished the session as the only driver not to set a time.
His team mate Jenson Button faired little better, completing only four laps.
Qualifying for Sunday's race starts at 0800 GMT on Saturday.
- Motor Racing
- Sports & Recreation
- Sepang Circuit
- Sebastian Vettel