Guptill had to abandon his innings initially on just three, with runners no longer allowed in international cricket, but he returned at the fall of the seventh wicket to get the hosts over the line in pursuit of 258.
Forty-one runs were still needed when he joined McCullum with four overs left. But the opener was to hit the winning run in the penultimate over to finish unbeaten on 27 as the Kiwis took a 1-0 lead with two more matches to play after this first one-day international at Seddon Park.
Jonathan Trott (68), Ian Bell (64) and Joe Root (56) had helped the tourists to a near-par total.
But Kane Williamson (74) and then McCullum (69no) kept New Zealand in it before Guptill's late heroics proved decisive.
