Substitute Darryl Duffy came off the bench to grab the second-half winner as Cheltenham overturned a 2-0 deficit to win 3-2 at Sixfields.
Early goals from top-scorer Adebayo Akinfenwa, with his 14th strike of the season, and an own goal from Robins defender Billy Jones gave Northampton a 2-0 lead.
But Cheltenham scored twice in three minutes to get back on level terms before the break through Darren Carter and Chris Zebroski before Duffy bagged the winner 10 minutes from the end.
Yates said: "After the first 10 minutes, in which we started awfully, I thought I'd picked the wrong team for this game! We'd worked on a lot of things in the two days before the game and all that could have been out of the window straight away.
"We spoke about (Adebayo) Akinfenwa and we spoke about the long throws but after 10 minutes Northampton had got a goal from both of them.
"Credit to the players though, they turned it around by sticking in there and manning up. We changed the system and we changed a few players and it wasn't with Tuesday's cup tie in mind.
"I wanted three combative midfield players and I wanted to get them to protect the two centre-backs, which I thought they did superbly well. We could have got a bit of a tanking but I think we got what we deserved in the end."
Northampton boss Aidy Boothroyd said: "We started very well and got on the front foot.
"We created lots of chances and put them under pressure. But towards the end of the first half we stopped doing all those good things and allowed Cheltenham back into it. We didn't expect them to come back but they did and you have to give credit to Cheltenham for that. We seemed to bring out the best in them."