Pompey led twice through Patrick Agyemang and Johnny Ertl, but McGlashan levelled twice, with his second arriving in the 92nd minute.
Yates hopes Cheltenham's last-gasp leveller has given them something to build on.
"We can't keep conceding goals like we are and having to dig ourselves out of trouble," he said.
"We didn't do enough and goals massively change games and mindsets.
"We started all right and then we go 1-0 down, fight our way back in, score a good goal and then we dominated, but we conceded again before half-time.
"It's getting ridiculous and we have to cut out defensive mistakes and bad decisions, everyone.
"We were the better team for last 20 minutes of the first half, but heads dropped when we went 2-1 down. We are not playing free-flowing football, but we showed a lot of fight.
"We have to do things better and we are still not there, but it's a start and I've told them well done for showing a bit of spirit, but we know things must improve."
Portsmouth had taken the lead through Agyemang in the fifth minute. He capitalised on Steve Elliott's stumble to poke the ball past goalkeeper Scott Brown from close range.
Agyemang should have been celebrating again in the 13th minute when he blazed over with the goal gaping after a fine run from Yassin Moutaouakil.
Portsmouth held the upper hand for most of the half, but the introduction of Russ Penn from the bench injected life into Cheltenham.
The quick winger rounded Sullivan and applied a neat finish inside the bottom right corner.
Cheltenham pressed for a second, enjoying their best period of the half, but Portsmouth regained the lead in the 43rd minute.
Jed Wallace reached the byline and pulled back for Ertl to tap into an empty net and it looked to be enough until McGlashan struck at the death.