The decision on Friday came after a growing number of storm victims, some runners, and other politicians criticised Bloomberg's decision earlier in the week to go forward with the marathon, one of the world's most popular sporting events. They said the race, expected to draw more than 40,000 runners, could have diverted police and other resources from recovery efforts.
Bloomberg, hours after he repeated plans for the race to take place, issued a statement saying the event had become a "source of controversy and division" and would be scrubbed. The race will not be run again until next year, organisers said.
The decision removes what could have been a dark spot on the mayor's legacy. Public opinion in the past few days had turned against the mayor, with growing numbers saying it was inappropriate to run the race when so many New Yorkers were suffering.
People angered by the marathon plans had set up online petitions calling for runners to boycott the 26.2-mile race, or to run backward from the starting line in protest.
The uproar grew after the New York Road Runners Club, the race organiser, set up generators in Central Park for communications and other operations. It said it had paid for those privately, not with public funds, but some complained that the equipment should have been donated to those without power, electricity or heat.
Some runners, hearing of the cancellation, expressed frustration.
"I have mixed emotions," said Christopher Miller, 34, of New Rochelle, New York, who would have been running his fourth New York City marathon. "Our hearts go out to people for their suffering, and also to the thousands who came from out of town and will leave without accomplishing what they set out to do months ago."
Another runner said the mayor should have stuck to the original decision, saying the race gives local businesses a boost and was set to raise large amounts for relief efforts.
"This was going to turn into a big recovery and healing event," said Usama Malik, 37, who works for a hedge fund. "I thought it was great that he (Bloomberg) made the decision to go on with it, to raise funds, to promote healing, and get people's minds off of everything else that's going on."
Sandy, which brought a record storm surge to coastal areas, killed at least 102 people after slamming into the US Northeast on Monday. Forty-one died in New York City, about half of them in Staten Island, which was overrun by a wall of water.
The marathon starts in Staten Island and weaves through all five of the city's boroughs. Hundreds of thousands of people line the streets to watch the race.
Run every year since 1970, the marathon attracts professional and amateur runners, and is so popular that organisers run a lottery system to determine who can compete.
The field features elite runners from around the globe, and is one of the six World Marathon Majors.
Among those who had been set to compete was Wilson Kipsang, the winner of this year's London Marathon, who had travelled 45 hours from his home in Kenya.
In announcing that the race had been called off, Bloomberg insisted it would not have diverted resources from the recovery effort. Hours earlier he had previously drawn parallels with the decision a decade ago not to cancel or postpone the marathon after the September 11, 2001 attacks.
However, he said, "we cannot allow a controversy over an athletic event - even one as meaningful as this - to distract attention away from all the critically important work that is being done to recover from the storm and get our city back on track."
Mary Wittenberg, the head of the New York Road Runners Club, said that as the controversy grew, she also was concerned about the reception runners may have received along the route.
At a news conference, Deputy Mayor Howard Wolfson said it had become clear that "something that every year brings joy and unity to this city had become divisive and painful, and this is a city that's had enough pain in the last week and I don't think we need to add more."