The decision was announced in a brief statement on the club website, which added that a new manager would "be appointed in due course".
Manchester United striker Ole Gunnar Solskjaer, currently manager of Norwegian club Molde, has emerged as the favourite following comments from Cardiff's honorary life president and former owner Sam Hammam.
He told BBC Wales: "He (Solskjaer) will have had very good basic grounding (at Molde) to what is needed now.
"We need a manager whose feet are on the ground and not just telling Vincent Tan, 'Give me your money,' as if all Vincent Tan is is a cash cow."
In the meantime, David Kerslake and Joe McBride will take temporary charge of team affairs.
Mackay was appointed as Cardiff boss following the sacking of Dave Jones after their Championship play-off defeat in 2011.
Former Watford boss Mackay led the Welsh side from the Championship to the Premier League, also taking them to a League Cup final, but his job had been under threat since the development of a disagreement regarding the club’s transfer policy following promotion.
Owner Tan gave Mackay a resign or be sacked ultimatum last week, but mediation from board members and supporter groups saw what appeared to be a reprieve.
Supporters had backed the Scot and criticised Tan, who also changed the club’s colours from blue to red against fan wishes, but the 3-0 defeat to Southampton proved to be the last straw.
Of his dismissal, Mackay said: "I leave with my head held high having gained a level of experience that, upon reflection, I suspect would have been difficult to find anywhere else in British football.
"The foundations laid here at Cardiff City will provide a solid platform for my successor and for the sake of the club I hope the progress that has been made so far will not be impaired in any way to today's enforced changes."
It is expected that supporters will continue to protest against the Malaysian businessman, who earlier in the season fired transfer boss Iain Moody and replaced him with an intern and family friend, citing a purported £15 million overspend in the summer.
Cardiff, still known as the Bluebirds despite their controversial kit change, lie 16th in the Premier League.
Former Manchester United and Norway striker Solskjaer is the favourite for the job, having reportedly met club executives in recent days.
Solskjaer's first and only managerial job has been at Molde, with whom he was won two Norwegian league titles and the Cup in three seasons. He had previously worked as a coach at United, for whom he scored 126 goals in 11 seasons as a player.
He had been linked with the Blackburn and Aston Villa jobs in recent years, but elected to stay in his homeland.