Grondona, who became AFA chief in 1979 and was senior vice-president of world soccer's governing body FIFA, died at the Mitre Sanatorium in Buenos Aires after suffering an aortic aneurysm.
He was rushed to hospital early on Wednesday but died shortly after midday before doctors could carry out surgery.
AFA vice-president Luis Segura has taken charge of the Argentine game's governing body and reports said the start of the league season scheduled for this weekend could be postponed.
"This is something very painful regardless of the differences one might have had (with him)," former Argentina coach Cesar Luis Menotti told broadcaster TyC Sports.
Grondona, who presided over Argentina's second World Cup victory in Mexico in 1986, rose to the highest position in the Argentine game from the presidency of leading first division club Independiente.
He and his son Humberto, Argentina's under-20 team coach, both said earlier this year Grondona would stand down in 2015.
"One's destiny is with God," Grondona was quoted as saying when asked how much longer he would remain in the position.
Grondona was on the verge of appointing the 10th national-team coach of his long reign. Alejandro Sabella was reported on Tuesday as having stood down after steering Argentina to the final of this year's World Cup on July 13.
- Sports & Recreation