But Yates admitted it needed some words from assistant manager Neil Howarth to bring the best out of the former AFC Wimbledon forward.
"I thought Byron was outstanding," Yates said.
"I thought he was shocking last week against Kidderminster Harriers and I told him so.
"In that last week, Neil has got into his head and done a lot with him.
"That is the player I signed, but he has to do it week in, week out now.
"I thought Terry (Gornell) was excellent and has been superb since the first day he walked in to the building.
"Some of the stuff he does for us is real quality.
"I want the strikers to play with a smile on their face and I've not seen Byron do that for a while."
A first-half strike from Gary Alexander and a two-goal blast from Nicky Adams had steered Crawley into a 3-1 lead with 20 minutes to go, Matt Richards netting Cheltenham's first-half effort from the penalty spot.
But Gornell and Harrison hauled the Robins back on level terms to take the tie into extra time before Harrison struck his last-gasp winner.
"After the first 20 minutes, when we showed them too much respect, we became a bit more aggressive and a bit more competitive," Yates added.
"They are a good side and have some good players.
"We had to dig in at times and Scott (Brown) has made some good saves. But we have had some good chances ourselves.
"To come back from two goals down, the guys showed courage and tremendous resilience.
"Burton showed us on Saturday that it can be done and it is nice to see that fighting spirit in the camp.
"If we are going to be successful you have to be together and resilient."
Crawley manager Richie Barker questioned some of his side's defending during the game, particularly central defender Kyle McFadzean, who was caught napping for Cheltenham's second goal.
"There was some poor decision-making and a lack of guts to put your head in the place where it might get hurt," the Crawley boss said.
"There was some fantastic football and some good goals. We always looked a threat going forward, but so did they.
"Kyle has been out for a long time and there's an element of rustiness, but he should be making better decisions than that.
"We are actually physically bigger than we were last year so it is a concern to concede from a set-piece.
"It is not hard to mark someone and pick them up and it's not tough that if it's in your area, you go and head it.
"It is something we need to address very quickly.
"We fizzed the ball around really well and created some good stuff, but they had as many shots as we did."