The fourth-seeded American, who has dropped only 12 games though her first four matches at Flushing Meadows, needed just 57 minutes to book her spot in the round of eight against former world number one Ivanovic.
"Andrea played really well today even though it was love and love," said Williams, who is riding a high from a London summer that brought her the Wimbledon title and the Olympic singles gold medal.
"I think I played really well today. I haven't played like that all tournament."
"I honestly don't think I started out strong in the tournament. I feel like today I'm getting more comfortable with the court and comfortable with the conditions," said the 30-year-old, who has lost just 12 games in four matches.
"I'm getting back (more) to my game, which is good. You know, I like to play better during the second week. Hopefully I can do that."
The 12th-seeded Ivanovic, 2008 French Open champion, beat Tsvetana Pironkova of Bulgaria 6-0 6-4 in her fourth-round match.
Ivanovic has been on a rollercoaster ride since reaching the top of the tennis world in 2008 but the 24-year-old Serbian is on the upswing after reaching the last eight at Flushing Meadows.
"It's amazing," the 12th-seeded Ivanovic said after reaching a Grand Slam quarter-final for the first time in four years.
"I have been here in the fourth round quite a few times, so to make that step and reach the quarter-finals, and also the first time since '08, I'm very thrilled."
Ivanovic, who used a potent forehand to register 18 winners against Pironkova, said it was hard to explain her fade after starting the 2008 season with a runner-up showing at the Australian Open followed by triumph at Roland Garros.
"Like a rollercoaster," she said. "There were some good moments and some tough ones, but it's a process. I understand it better now.
"You can't have everything at the same time. You have to be consistent on practice courts and then in the matches. I felt I had that this year."
Ivanovic has lost all three previous meetings against Williams, including twice at the US Open. Last year, she fell 6-3 6-4 to the American in the last 16 at Flushing Meadows.
"I put myself lots of times in opportunities for the big matches and the big wins, and I never managed to do that. Now I have another chance, and I really hope I can pull it together this time," Ivanovic said.
Ivanovic said she has gained strength from childhood friend Novak Djokovic, who last year surged to the world number one with triumphs at the Australian Open, Wimbledon and US Open.
"We did speak a lot. He obviously had a great year last year. He's a dear friend of mine. It's great to get some experience from someone like him," said Ivanovic.
Ivanovic, who has recovered from a foot injury to stage her grand slam revival in New York, said she relished the chance to play Williams for a spot in the semi-finals.
"She's obviously a great player and she's been playing the best out of all the girls on tour at the moment," Ivanovic said.
"Her serve is definitely her strength. I really have to be sharp from the first moment on and stay aggressive and stay in the moment and try to enjoy it as well and enjoy the challenge."