Yannick Bolasie put the visitors ahead on 13 minutes, but it was Speroni's save from Conor Sammon's 86th minute penalty that saved the three points for Palace.
In the night's other match Bakary Sako scored an injury time equaliser to give Wolves a 1-1 draw at home against Watford.
Almen Abdi's strike just before half-time appeared to have given the Hornets a victory, but Sako popped up at the death to deny them.
At Pride Park Speroni saved Sammon's late penalty as Palace secured a first Championship away win since November with a 1-0 victory against Derby.
Bolasie clinically scored the game's only goal for the promotion hopefuls, who looked good value for the three points until captain Mile Jedinak fouled substitute Ben Davies in the penalty area with three minutes remaining.
However, Sammon's powerful spot-kick was brilliantly deflected onto the bar by the Argentinian shot-stopper to consign Nigel Clough's side to a first home defeat of 2013.
Palace manager Ian Holloway received a welcome boost with Wilfred Zaha, on loan from Manchester United after joining the Premier League leaders in January, returning from a dead leg that had kept him out of Saturday's defeat at Sheffield Wednesday.
Zaha had to wait just five minutes for his first opportunity - a long ball upfield was flicked on by Glenn Murray and a change of pace allowed him to sear in behind the Derby backline, only to shoot straight at Adam Legzdins.
Palace were enjoying the better of the opening exchanges and Murray's lay-off for Stephen Dobbie might have brought the opening goal had the former West Brom man's 20-yard shot not deflected into the side-netting.
The breakthrough came in the 13th minute in freak circumstances. Eagles skipper Jedinak won a thunderous challenge inside his own half and the ball went flying up field to Bolasie, who took full advantage of Derby's AWOL backline to coolly round the keeper and fire home.
Palace might have had a second three minutes later when Jedinak's 25-yard thunderbolt cannoned off the crossbar, while at the other end only the excellent Damien Delaney's last-ditch challenge prevented Sammon from going clean through on goal.
Derby began to play their way into the game - Jamie Ward had a volleyed effort parried behind by Speroni, before John Brayford's low cross was only half-cleared to Craig Bryson, who shot straight at the veteran keeper.
A delightful one-two between Paul Coutts and Chris Martin put the former clear down the right, and after his low cross had deflected towards the penalty spot Norwich loanee Martin drew a good save from Speroni.
A quick visiting counter attack five minutes before half-time saw Bolasie play in Dobbie, but the Scot sliced his shot wide.
Another swift break saw Bryson deny Bolasie a second when he headed the Frenchman's 44th-minute effort off the line with Legzdins in no-man's land.
Derby were unlucky not to be awarded a penalty in first-half stoppage-time when Peter Ramage appeared to haul Martin down inside the box, but Sammon should have levelled seven minutes after the break when Martin's clever overhead kick played Michael Jacobs in down the left channel and the substitute squared to the Irishman, only for Speroni to smother after the former Wigan forward hesitated.
Legzdins did well to deny the visitors a second goal in the 65th minute when he brilliantly tipped Jonathan Williams' 20-yard effort over the bar, after the substitute had gone on a dazzling run from the halfway line.
Derby were enjoying the better of the second half, but when the ball dropped to Jacobs unmarked 12 yards from he wastefully dragged his volley wide.
The hosts were struggling to fashion clear-cut opportunities, but Speroni had to be alert to push away Richard Keogh's low drive from the edge of the box, before Sammon's penalty woe capped a miserable night for the home fans.
At Molineux, Sako's 93rd-minute equaliser salvaged a 1-1 draw for ailing Wolves and put a dent Watford's promotion charge.
Manager Dean Saunders, the club's fourth boss in 12 months, is still waiting for his first victory in charge having overseen nine of their run of 13 winless matches, with Wanderers - in the npower Championship drop zone - still facing the very real prospect of successive relegations, as they experienced in the mid-1980's.
Matters are not much rosier off the pitch with chairman Steve Morgan this week insisting he will leave the Black Country club if that is what supporters want, although a pre-match protest aimed at the owner and under-fire chief executive Jez Moxey struggled to really grow in number beyond 50 fans.
Yet Sako's well-taken injury-time equaliser drew welcome celebrations from a frustrated Wolves crowd, who will still believe in survival.
The game's defining moment of brilliance looked to have came just before half-time as Watford playmaker Almen Abdi, one of a contingent of players on loan from Udinese, converted a delightful free-kick for his 10th goal of the season.
The Hornets could at that point thank goalkeeper Jonathan Bond, making his full league debut for the Hertfordshire outfit, for still being on level terms having made an impressive double save.
It was a point towards in-form Watford's march to a Barclays Premier League return, although Gianfranco Zola's second-placed side will certainly see it as two dropped.
All the stats bar one pointed to an away win before kick-off, with Watford having claimed just one win in their previous 15 visits to Molineux.
Saunders, with Kevin Doyle and Stephen Hunt back in his starting XI, reverted to a more familiar 4-4-2 formation from the off and it was his side who started with the greater intent.
Watford soon regained a foothold on proceedings, though, Matej Vydra seeing an effort deflected narrowly wide.
The first clear chance of the game arrived in the 19th minute as Doyle found possession on the edge of the area and saw his effort well parried by Bond, in for the injured Manuel Almunia, who got up quickly to superbly keep out Bjorn Sigurdarson's rebound.
Marco Cassetti fired straight at Carl Ikeme following Jack Robinson's mis-control and as the game was drawing towards stalemate at half-time, the visitors broke the deadlock.
Vydra drew a soft foul from Wanderers skipper Karl Henry and Abdi, 20 yards out from a central position, floated a super free-kick over the wall.
The home crowd's frustration seemed to be growing at the start of the second period, although Doyle's surging run and shot brought some positive respite.
Watford should have doubled their advantage in the 66th minute but Ikechi Anya somehow failed to convert after being teed up by the impressive Cassetti, with Cristian Battocchio drilling narrowly over from distance minutes later.
Wolves wasted a chance to draw level in the 77th minute, Sigurdarson seeing his shot saved by Bond before Robinson was well off target with the rebound.
Yet, with the game seemingly beyond them, Sako popped up to convert from Matt Doherty's long throw-in.
Watford substitute Fernando Forestieri still had a chance to snatch matters but sliced wildly off target from close range.
