It has now become something much worse.
With the emergence of reports filled with damning evidence of wide-scale deaths among mistreated migrant workers brought into Qatar to work on World Cup-related building projects, it became clear the sport's governing body FIFA now has more to worry about than being seen as just sleazy.
Unless various human rights organisations, a detailed investigation by London's Guardian newspaper and official statistics from the government of Nepal (where most of the migrants came from) are all wrong, then FIFA has blood on its hands.
It is the blood spilled on building sites across Qatar, where even nine years out from the tournament's kick off a frantic race is on to sandwich nine stadiums, a renovated freeway system and sufficient hotel space into a region smaller than Connecticut and with a population of two million.
It is the blood of human "slaves", according to the Guardian report, which detailed savage abuses of basic human rights on a sickening scale. As if the deaths of 44 Nepalese workers between June 4 and August 8 were not enough, it also told of workers being denied access to free drinking water in the searing Qatar heat, unpaid salaries, non-existent medical care and confiscation of passports to prevent employees from leaving.
With its thirst for rapid building expansion related to the World Cup, Qatar, according to the International Trade Union Confederation, has become a "slave state."
FIFA's executive committee is meeting in Switzerland this week, with a discussion on the 2022 World Cup on Friday's agenda. Nearly three years on from its great folly of awarding the event to Qatar in the first place, it now more than ever has a duty to act.
That bidding process back in 2010 will go down in history as the defining moment when football's money madness reached a new low, when free-flowing cash falling into grubby hands threw reason and propriety out of the window.
Qatar's successful bid, beating competition from the United States, South Korea, Japan and Australia, was never truly a feel-good tale of a tiny state overcoming the odds, even though that's what its oil-rich ruling family would like you to believe.
Half of the men who voted in its favor are under investigation for corruption. FIFA president Sepp Blatter himself admitted two years ago a vote-trading pact existed between the Spain/Portugal 2018 and Qatar 2022 bids, while Qatari investment in various nations with vote-carrying members led to insinuations of political influence being used to manipulate the decision.
The governing body is stuck with that perception of malfeasance, but right now FIFA does wield the power to effect change in relation to the plight of workers with little alternative protection.
The threat of removing the tournament that Qatar spent, sorry, worked so hard for, would be the ultimate carrot for the country's rulers to launch a crackdown on exploitation, as many of the Nepalese workers brought in incur heavy debts to shady recruitment "agents", effectively tying them to their new jobs or risking financial ruin.
And Qatar knows full well that any of its defeated opponents, the US in particular, would jump at the chance to step in as substitute.However, a "quick fix" of immediately taking the tournament away and switching it to another country would not be the right move here. For a start, such an action would merely worsen the plight for those workers affected; building projects would be put on hold and many would likely be left unpaid and homeless in a foreign country.
FIFA made this thorny bed for itself and now needs to lie in it, never mind the scratches. At some point it has to start making decisions for the right reasons, and this is as good a place to start as any.
A firm and genuine warning that continued human rights abuses will lead to removal of World Cup privileges is the most practical and sensible option, one that will hopefully have at least some kind of positive humanitarian effect.
Until now, the biggest controversy surrounding Qatar 2022 was related to its searing summer temperatures that average over 100 degrees Fahrenheit, prompting calls for a shift to a winter schedule.
When compared to human life, the logistical headaches a time-frame move would cause seem somewhat trivial by comparison.
The most pressing task for soccer right now is to turn up the heat … on Qatar.