The U's followed up their 4-1 thrashing of Portsmouth at Fratton Park with a performance of resilience and skill at home, and it was two set-piece goals which brought them victory.
Danny Rose put them ahead in the 33rd minute with a free-kick from 25 yards and, after Tom Soares had equalised for the Shakers after 70 minutes, Johnny Mullins restored Oxford's advantage by turning in David Hunt's low free-kick 10 minutes from time.
"It's been a great week for us in terms of the league," said Wilder.
"We're not a set-piece team as such, we try to pass it and move teams around and play that way. But we do work on set plays and when they come off it's great. At set pieces you have to have people who come alive and you have to have good delivery, which we have. We work on all that stuff and we got the benefits."
"There's still a lot of improvement in the team. I didn't think we started the second half on the front foot, which disappointed me, and they took advantage of that and got themselves back in the game. Then we came alive again.
"We had to make sure we were right defensively because we've watched Bury a couple of times and they are good going forward. At times in the first half we got into really good positions."
Bury had Nathan Cameron sent off, he received a second yellow card for protesting too much. He had given away the free-kick which led to Mullins' winner, collecting a yellow card for the initial foul before being booked again for arguing when Oxford turned it into a goal.
Wilder added: "I'm not surprised someone got sent off because there were a lot of fouls out there and Bury are a tough old side to play against. I don't think the referee got any of the big decisions wrong."