The US-born Italy striker, back this season after successive knee injuries effectively sidelined him for two years, took his tally to eight goals in as many games while Juventus coach Antonio Conte was left visibly seething by his team's collapse.
There was plenty of other drama with a total of eight penalties being awarded in Sunday afternoon's six games and a first top-flight win for Serie A debutants Sassuolo who beat Bologna 2-1.
Sampdoria converted two penalties, the second in the sixth minute of stoppage time, for a 2-1 win at Livorno while Jorginho scored two penalties for Hellas Verona as they came from behind to beat Parma 3-2.
"We suffered a 10-minute nightmare," Conte told reporters. "I thought it was all over bar the shouting. We dominated the game for long periods and it all went down the drain in 10 minutes."
Juventus had dropped only two points all season in the league and appeared to be cruising when a controversial Carlos Tevez penalty and a Paul Pogba goal gave them a 2-0 halftime lead.
Tevez collapsed dramatically after minimal contact from Gonzalo Rodriguez before dusting himself down to fire home a penalty in the 37th minute.
Three minutes later, Fiorentina's Juan Cuadrado slid in and sent an attempted clearance looping across the face of his own goal, where it landed at the feet of Pogba who tapped in the ball.
Fiorentina's fightback began with another controversial penalty in the 66th minute when Matias Fernandez was bundled over and Rossi scored from the spot.
Juventus then collapsed, conceding three goals in five minutes. Rossi levelled in the 76th minute when he collected the ball on the edge of the area, beat two defenders and fired past Gianluigi Buffon.
Two minutes after that, the Juventus defence left Joaquin Sanchez all alone and the Spaniard swept home the equaliser. Then Colombian international Cuadrado led a counter-attack and slipped the ball to Rossi who drove home his third.
Juventus were left in third place with 19 points, behind Napoli on goal difference and five points adrift of Roma. Fio
In the late game, Torino grabbed a point with a last-minute free-kick when holding Inter in a 3-3 thriller.
Inter lost keeper Samir Handanovic to a red card early on following a foul on Alessio Cerci, who had the subsequent penalty saved. However Alexander Farnerud did opening the scoring for the Turin club.
Fredy Guarin converted a spectacular effort to draw Inter level just before the break and, after Ciro Immobile had put Torino in front again, Rodrigo Palacio scored twice before 71 minutes to put Inter in control.
However, Nicola Bellomo’s free-kick beat keeper Juan Pablo Carrizo right at the death.