With Milan safely through to the Champions League group stage after Wednesday's 3-0 win over PSV Eindhoven, Allegri, who has clung on to the job for three years, complained of "pontificating from the outside".
"For two years, there has been talk about the next coach of AC Milan. People have to be more objective," Allegri, whose side host Cagliari in Serie A on Sunday (1845 GMT), told touchline reporters after the Eindhoven match.
"There is too much pontificating from the outside, there should be more respect for other people's work."
"I'm happy about the way the lads played but I'm also a little angry," added Allegri, referring to media speculation that he would be chopped if Milan failed to make the Champions League group stage.
"People on the outside are trying to make me angry but it doesn't work."
Allegri, initially a surprise appointment, won the Serie A championship in his first season at Milan followed by second place in 2011/12.
Last season, however, was tougher as Milan tried to balance the books and rebuild their team with younger players.
They made an awful start to their campaign but recovered well in the second half, especially after Mario Balotelli joined from Manchester City, to finish third and clinch a place in the playoff.
Despite that, there was more talk of Allegri's departure during the summer, until he received the public backing of the Curva Sud (South Curve) ultras, Milan's influential group of hardcore supporters.
Allegri even gave Italian media a minor scare when he joked about quitting on Wednesday.
"I'll take the night to think about it," he told Mediaset Premium. "Maybe, I will go to Livorno by the sea, go out in a boat and just watch the odd match at San Siro."
Despite the 4-1 aggregate win over the Dutch, Milan badly need three points on Sunday after beginning their campaign with a 2-1 defeat at promoted Hellas Verona, where the defence showed the same frailties as last season.
Having spent most of last season playing catch-up, Milan will not want a repeat this time, especially as they and Napoli are seen as the only teams capable of launching any sort of challenge to the dominant Juventus.
Champions for the last two seasons, Juventus host Lazio on Sunday, just two weeks after thumping the Coppa Italia holders 4-0 in the season-opening Supercup match. Both teams won their opening league games last week.
Napoli, who began with a convincing 3-0 over Bologna, visit Chievo (Saturday 1600) with striker Gonzalo Higuain looking for his first goal since his 37 million euro move from Real Madrid.
Higuain is expected to recover for the game following his boating accident on Monday, when he slipped during an excursion to the island of Capri and needed 10 stitches in a facial wound.
Germany forward Mario Gomez, who joined Fiorentina from Bayern Munich, will also want to get off the mark for his new side as they travel to Genoa (Sunday, 1845). Fourth last season, Fiorentina sold striker Adem Ljajic to Roma on Thursday, the second major departure following Stevan Jovetic.
Serie A debutants Sassuolo will attempt to pick up their first-ever top flight points at home to Livorno (Sunday 1845), promoted with them last season, and AS Roma's host of new signings take on Hellas Verona (Sunday 1600).