Armstrong on Thursday said he would no longer fight charges by the U.S. Anti-Doping Agency, which quickly said it would strip him of his titles and ban him from competitive cycling .
"He had the right to rip up those charges but he elected not to, therefore the only interpretation in these circumstances is that there was substance in those charges," Fahey told Reuters in a telephone interview from his Australian residence on Friday.
"I'm not going to attempt to understand why he's done that ... I can only take it as it stands - that it leads only to the conclusion that he is a drug cheat.
"My understanding is that when the evidence is based upon a career that included seven Tour de France wins then all of that becomes obliterated."
Texas-born Armstrong, who retired from professional cycling last year but remains the face of his anti-cancer charity Livestrong, has long denied using performance-enhancing drugs.
He maintained that stance in a statement on Thursday, stressing that there was no physical evidence to support what he called "outlandish and heinous claims".
Armstrong, who has never failed a doping test, said he would jump at the chance to put the allegations to rest once and for all, but refused to participate in the USADA process, which he called "one-sided and unfair".
Australian Fahey dismissed Armstrong's criticism of USADA and said he had found the doping authority to be a "very professional organisation" that had followed the rules in the cyclist's case.
"They have acted on rules which are compliant with the WADA code," Fahey said. "Now Mr. Armstrong has said: 'Enough, I'm not going any further' and he's endeavoured to shoot the messenger.
"I think that's disappointing ... He's never gone before a tribunal. There's never been a hearing related to this behaviour, so what is he tired of?"
Fahey said the issue was now for USADA to make a formal decision with sanctions attached, and said the weight of evidence against Armstrong, and his refusal to respond to the charges, made the allegations "factual".
"I think you've got to look at the facts and the evidence that has only been collected in the last couple of years.
"There are 10 riders and several other witnesses with evidence (against him).
"There can be no other interpretation. To refuse the charges can only leave the interpretation that he is a cheat."