Cheltenham dominated the opening stages with Billy Jones having a free-kick deflected for a corner while Steve Elliott and Kaid Mohamed were denied by keeper James Bittner.
The pressure finally paid off on 16 minutes when a slick pass from Marlon Pack found Jermaine McGlashan down the right and his low cross was neatly flicked home by Shaun Harrad. Jake Taylor glanced a header wide as the Robins continued to push but, after being on the back foot for so long, Hereford levelled with their first attack four minutes later.
Ryan Bowman's cross found Josh O'Keefe and although his initial header was pushed out by keeper Scott Brown, O'Keefe powered in the rebound.
The second half was more even with both sides having chances to snatch the game.
Mohamed twice tested Bittner with long-range efforts while Will Evans had an effort tipped over the bar at the other end.