With the game goalless early in the second half, Aaron Ramsey raced on to Olivier Giroud's through ball with Swansea right-back Dwight Tiendalli in pursuit.
The Welshman went to ground but referee Howard Webb waved away the Arsenal appeals, and replays appeared to suggest Ramsey had clipped his own heels. But Wenger firmly believed his side should have been awarded a spot-kick.
"It was a penalty and the referee saw it," he said. "Do you know why he did not give it? Because he thought the guy had not done it on purpose, and I also think it was accidental.
"But he (Ramsey) did not trip himself, he was caught clearly by the leg of the Swansea player. The referee hesitated and he did not give it."
Michu, who settled last month's league meeting between the sides, had looked to again be Arsenal's downfall as he gave Swansea the lead just 73 seconds after coming on as a second-half substitute.
But two goals in three minutes from Lukas Podolski and Kieran Gibbs put the Gunners on course to advance, only for Danny Graham, who scored a late leveller against Aston Villa in midweek, to rifle home with three minutes left.
Wenger felt his side should have seen the game out after leading late on.
"We should have won this game, the first half was a bit locked and one-paced but the second was much more open," said the Frenchman. "We created many chances and found ourselves 1-0 down, but we showed character and quality.
"Unfortunately I felt on the corner just before their goal, when we conceded it we lacked some calmness and focus. It is frustrating to concede a goal like this when we were first to the ball but didn't clear it, but we showed heart and it is good to repeat that."
However, Swansea boss Michael Laudrup said his side deserved a replay.
"I think the overall feeling is that it was a fair result," said Laudrup. "In the second half we made a couple of changes that paid off with Michu's goal, but to be fair to Arsenal they put us under pressure, they kept the ball, took over possession and in those 15 minutes we lost the ball too much.
"Then they scored two very late goals, but again we came back and we again showed the character of our team to score the equaliser."