The Republic of Ireland international won a first-half penalty when he went down under a challenge from Joey O'Brien and then added further to Allardyce's ire by stepping up to put away his fourth of the season.
Allardyce felt the former Manchester United junior had influenced referee Kevin Friend by going down too easily as O'Brien put his hands on him and was not afraid to say so afterwards.
He also told his own players they were responsible for the loss too - poor finishing their greatest crime - while another decision not to give a second penalty for an alleged handball by Jake Livermore also got his back up.
"Not finishing off our chances was a big problem for us," he said.
"The ultimate failing we have away from home is not turning our chances into goals.
"Again that cost us something out of the game, irrespective of the two poor decisions the referee has given. There were two really poor decisions, but we still should have got something by capitalising on the chances we created.
"That's the disappointing thing for me and is a massive problem for the players and their responsibility to be clinical."
On Friend he added: "The other side is two very poor decisions in terms of major decisions by the referee. One, Brady takes a simulation on a slight contact with Joey O'Brien when the ball is not near him or the goal. He buys the penalty and the referee falls for it.
"Livermore puts his arm out and knocks the ball for a corner but the referee doesn't see it and doesn't give us a penalty. So it's our responsibility for not finishing our chances and the referees as well for us losing the game."
The remaining narrative of Allardyce's assessment was centred on his side's anaemic final-third play.
They have failed to win in the Premier League since the opening day as a consequence of it and have five points from six games.
"It's a worrying trend but it's not as if we've gone away from home and not created chances," he said.