A 2-1 third-round defeat by Ole Gunnar Solskjaer's Cardiff at St James' Park on Saturday saw the Magpies slip out of the competition in their opening fixture for the third time in four seasons under manager Alan Pardew.
Indeed, a 2-1 third-round victory over Blackburn in January 2012 remains the 52-year-old's only win in the competition, and that proved something of a false down as his side succumbed at the next time of asking at Brighton, who also removed them last season with League Two Stevenage having previously claimed their scalp.
The club's failure to progress in the cup they have lifted six times, but not since 1955, is a sense of intense frustration for the fans, some of whom have accused them of prioritising the Barclays Premier League at its expense, although Taylor insists that is not the case for the players.
Asked if he understood the disappointment of fans who greeted the final whistle with boos, he said: "Yes. It's a killer blow.
"We understand. Listen, this competition is massive - people don't realise how big this competition is.
"Even from a young lad, come the FA Cup, this is what you want to be in and to try to win.
"But the last three years just haven't been good enough. We just need more belief."
Newcastle, who made seven changes to the side beaten at West Brom on New Year's Day, shaded a tepid first-half despite Cardiff defender Mark Hudson having a goal chalked off for a foul, and were unfortunate not to take the lead when Hatem Ben Arfa rattled the post with a left-foot drive.
They stepped up a gear after the break and did get their noses in front with 61 minutes gone when Papiss Cisse claimed his third goal of the season from close range after Moussa Sissoko had been played in over the top by Yoan Gouffran.
However, with the game there for the taking, the Magpies fatally relaxed and allowed substitutes Craig Noone and Fraizer Campbell to take over, the former smashing home a stunning 73rd-minute equaliser from distance and the latter winning it with a towering header seven minutes later.
Taylor said: "We are very disappointed. This is the third year on the trot for us - that's got to hurt.
"We didn't deserve to win. Maybe for 20 minutes in the first half, we played well and possibly 10 there in the second half, we had a good little stint, but they just wanted it that bit more.
"Set-pieces win you games and we know that over the years that we have been in this competition. We didn't deal with the set-plays and at the end of the day, that cost us the game."
Pardew's men now have only the league and the quest for European football - something which placed damaging demands on their resources last season - for which to play.
They resume next weekend when title challengers Manchester City head for Tyneside to face a side which has lost its last three games in all competitions.
Taylor said: "The next game, Manchester City, a team that's challenging for the title - like we have proven before, we have beaten the top sides, so they are going to come here and they are going to have a difficult game.
"It's a kick in the backside for us, but we will bounce back and this week, we will work hard for the game against Manchester City."
For Taylor, there was at least the satisfaction of returning to senior action for the first time since his sending-off at City during the opening weekend of the campaign with injury having taken its toll since, although that proved scant consolation.
He said: "It's been great being out there and getting that opportunity again. There's no better place, in front of those Geordies, so it was great for me to be out there.
"But I am just devastated we lost."