Tokyo beat Istanbul by 60 votes to 36 in a head-to-head vote by IOC members in Buenos Aires on Saturday, giving the Japanese capital the Games for the second time.
Madrid had been eliminated in a first round of voting.
"Both Tokyo and Madrid have hosted the games before; Istanbul hasn't. It hasn't been fair," Erdogan was quoted as saying in Turkish media.
"In a way, they are cutting ties with the 1.5-billion-people Muslim world."
Civil unrest, the unstable political situation on the country's doorstep and a wave of high-profile athletics doping cases are seen as the chief culprits for the IOC's decision to overlook Turkey, which has a predominantly Muslim population, again after Istanbul failed in bids to land the 2000, 2004, 2008 and 2012 Games.
While the unrest in neighbouring Syria was seen by some as counting against the bid, others felt a heavy-handed police crackdown during recent anti-government protests also damaged Turkey's image.
Tokyo, which hosted the 1964 Games, had an estimated non-Games budget of around $4.4 billion for 2020 plus $3.4 billion for the actual event.
Istanbul's proposal had a total cost of $19 billion, making it more ambitious but also risky given the country's lack of experience in staging major sports events.
Another worry for Istanbul has been the wave of doping cases which have resulted in the Turkish Athletics Federation banning dozens of athletes for drugs violations, most recently double European 100m hurdles champion Nevin Yanit.
Turkey's Sports Minister Suat Kilic said doping was not an issue peculiar to Turkey while Erdogan said the country was taking steps to fight it.
"We have said 'zero tolerance against doping' and have started our work," Erdogan said.