It was the Atletico star's 27th goal of the season and saw him come out on top in a face off with Milan's enigmatic forward, Mario Balotelli, whose lively performance was prematurely ended by a shoulder injury in the second half.
The defeat was hard on Rossoneri coach Clarence Seedorf, who twice saw his team hit the woodwork in the first half of what was his Champions League bow as a manager.
The hosts had the better of the chances and could have gone ahead on 15 minutes when Kaka's left-foot drive deflected onto the crossbar, with Thibaut Courtois beaten.
At the other end Costa crashed an acrobatic attempt over the top and Raul Garcia saw two efforts beaten away as the visitors ended strongly and it looked all set to end in stalemate until Abate's error gifted Costa the opening and the striker made no mistake with a bullet header.
KEY MOMENTS PRE-MATCH: Taarabt starts, Villa on the bench - QPR loanee Adel Taarabt is handed a Champions League bow by Seedorf. The mercurial playmaker supports Kaka and Balotelli in attack. Atletico make the one change with Miranda coming back in. 26-goal Costa leads the line with David Villa on the bench.
14' - OFF THE BAR! Kaka picks up the ball on the right and unleashes a stunning effort that is deflected onto the bar with Courtois beaten.
19' - WONDER SAVE! Courtois shows just why he is rated as one of Europe's best keepers brilliantly tipping Andrea Poli's header onto the post after a great ball in from the right by Taarabt. 30' - CLOSE! Balotelli slips a delicious flick into the path of Kaka who curls a shot just over the top from the right side of the area. Milan carving out the better chances.
51' - ACROBATIC EFFORT! Costa pulls away to the far post and attempts an overhead kick that sears over the crossbar.
67'- CLOSE! Essien rises to meet a right-wing corner but he powers his header a few yards wide of the far post.
76'- CHANCE FOR ATLETICO. Abbiati is forced to deny Garcia's rasping low drive following a lightning counter.
78' - BLOW FOR MILAN: Balotelli goes off clutching his shoulder in pain. He's replaced by Giampaolo Pazzini.
83' - GOAL: AC Milan 0-1 Atletico Madrid. Costa thumps a sensational far post header into the corner after Abate inadvertently flicked a corner into his path. 87' - SO CLOSE! Rami crashes in a thunderous 35 yard free kick that arrows inches past the post.
87' - SO CLOSE! Rami crashes in a thunderous 35 yard free kick that arrows inches past the post.
TALKING POINT: In Courtois and Costa, have Atletico the recipe to go far? The big Belgian continues to illustrate why Atletico are desperate to keep hold of him and the end of this season will be a key crossroads in his career. Petr Cech isn't getting any younger and Courtois would be an ideal long-term replacement at Stamford Bridge. The question is whether Chelsea would be willing to sacrifice the young stopper in their rumoured efforts to sign Costa? Once again the Brazilian was their match winner and he would no doubt add something extra to Chelsea's attack. But at the moment they are very much part of a strong Atletico side who are dark horses to go all the way in this tournament with a strong defence and lethal attack.
Milan have won only one of their last nine home games against Spanish visitors.
Atletico are well on course for a first quarter-final appearance since 1997, when their coach Simeone was playing for the Rojiblancos.
Atletico are only one of three teams undefeated in the group stage and are unbeaten in their last five away games in UEFA competition.
MAN OF THE MATCH
Diego Costa (Atletico) - Ploughed a lone furrow up top as AC Milan bossed the first half. His hold up play offered key moments of respite and he typically popped up with the winner when the Spaniards turned the screw late on.
PLAYER RATINGS -
AC Milan: Abbiati 6, De Sciglio 6, Rami 6, Bonera 6, Essien 6, Poli 7, Emanuelson 6, N. de Jong 6, Kaka 7, Taarabt 7, Balotelli 7. Subs: Abate 6, Pazzini 6, Constant 5.
Atletico Madrid: Courtois 8, Godin 7, Juanfran 7, Insua 6, Miranda 6, Mario Suarez 6, Koke 7, Raul Garcia 6, Arda Turan 6, Gabi 6, Diego Costa 8. Subs: Rodriguez 6, Lopez 5.
THE MATCH IN A TWEET