Casey made his comments during the postgame news conference after his team's 93-92 loss to the Atlanta Hawks on Wednesday at Philips Arena.
In the closing seconds of the game, Toronto's Alan Anderson missed a jump shot but his team mate DeMar DeRozan grabbed the rebound before his attempt under the basket was blocked by Atlanta's Al Horford.
Casey then charged on to the court demanding an explanation for why no foul was called on Horford for apparent contact on DeRozan.
"I'm tired of this," Casey said during the news conference. "I'm tired of losing games because of missed calls at the end of the game. I know the league's going to come down on me for this, but I don't care."