Alex Pearce restored Reading's lead on 56 minutes and McCleary's 25-yarder meant his team had some breathing space after former R's defender Kaspars Gorkss was sent off for a 71st-minute challenge on Junior Hoilett.
It meant back-to-back defeats for Harry Redknapp's pre-season title favourites, who remain four points behind second-placed Burnley.
They are clearly missing sidelined top scorer Charlie Austin and must hope the imminent arrival of Ravel Morrison on loan from West Ham can help revitalise their flagging season.
The visitors, meanwhile, cemented their place in the top six with the perfect response to their recent loss at home to Sheffield Wednesday.
Reading went ahead in the 10th minute when Williams found Adam Le Fondre out on the left and then drifted into the box unmarked to head home the striker's cross.
QPR hit back 13 minutes later after the Royals were unable to clear their lines following Armand Traore's cross, Doyle following up to score after Will Keane's effort had been parried by keeper Alex McCarthy.
The equaliser galvanised the home side, who went close to going ahead when Gorkss nudged Hoilett's low cross against the outside of his own post.
And from the resulting right-wing corner from Benoit Assou-Ekotto, striker Keane - making his debut following a recent loan move from Manchester United - headed narrowly over.
But two Reading goals in the space of as many minutes early in the second half left Rangers shell-shocked.
Pearce, playing because his red card against Wednesday was rescinded after Reading appealed, headed in from Jobi McAnuff's corner.
And McCleary collected the ball in his own half before galloping towards the penalty area and unleashing a brilliant 25-yard shot that flew past goalkeeper Rob Green and into the top corner.
Reading continued to press and wanted a penalty for a handball against Clint Hill before then having to withstand a spell of pressure.
Doyle was a yard out when he failed to connect with Hoilett's cross and Traore had a shot saved by the impressive McCarthy.
Nigel Adkins' side needed to defend resolutely after Gorkss was shown a straight red card for clattering into Hoilett on the right flank.
They got men behind the ball and Joey Barton's shot over from 20 yards was as good as QPR could muster in the final minutes.
- Sports & Recreation
- Kevin Doyle
- Kaspars Gorkss
- Junior Hoilett
- Alex Pearce
- Armand Traore