American Horner, who turns 42 next month, attacked close to the finish of the agonisingly steep climb to finish sixth behind stage winner Vasil Kiryienka of Belarus and gain 25 seconds on Nibali, who placed 10th.
With three stages to go until the race finishes in Madrid on Sunday, Nibali leads Horner with Spaniard Alejandro Valverde third at 1:09.
"Chris is doing well. He's got an extra gear compared to the rest of us, and we've still got two very difficult mountain stages to come," Giro d'Italia champion Nibali said.
Nibali tried to stay with Horner as the American attacked Pena Cabarga's steepest slopes of 21 per cent, ignoring an earlier drive by Spain's Joaquim Rodriguez, but said it had proved impossible.
"Horner was so strong there and going at such a pace, he didn't just drop me, he dropped all the favourites," the Astana rider added. "He was going up the climb at a pace that was impossible to follow."
Asked if he was surprised by the level of Horner's performance given his age, Nibali, 28, responded with a laugh: "I think when I'm 42 maybe I won't be racing.
"Horner's shown he's got great physical condition and if he doesn't have a crisis in the last few days he's going to be very difficult to beat."
The showdown between the top two is likely to come on Saturday's ascent of the Angliru, Spain's toughest single climb.
"I'm looking forward to it. It's going to be a big battle," RadioShack-Leopard rider Horner said.
"I had amazing legs today. At some stage towards the end, I raced a little safe today."
Having been part of an early attack on the 186-km stage to Spain's northern coast, Kiryienka pulled away 33km from the line to make a lone triumphant assault of Pena Cabarga and finish 28 seconds ahead of Denmark's Chris Anker Sorensen and Australian Adam Hansen.
"It was a really hard climb, but I tackled it like a time trial, trying to keep a steady pace and that seemed to do the trick," Kiryienka said.
Results from Vuelta a Espana (Burgos - Pena Cabarga, 186.5 km) on Thursday
1. Vasil Kiryienka (Belarus / Team Sky) 4:46:48"
2. Chris Sorensen (Denmark / Saxo - Tinkoff) +28"
3. Adam Hansen (Australia / Lotto) +1:18"
4. Martin Kohler (Switzerland / BMC Racing) +1:34"
5. Egoi Martinez (Spain / Euskaltel) +1:42"
6. Chris Horner (U.S. / RadioShack) +1:53"
7. Amets Txurruka (Spain / Caja Rural) +2:02"
8. Joaquim Rodriguez (Spain / Katusha) +2:13"
9. Alejandro Valverde (Spain / Movistar) +2:14"
10. Vincenzo Nibali (Italy / Astana) +2:18"
11. Nicolas Roche (Ireland / Saxo - Tinkoff)
Classification from Vuelta a Espana after Stage 18 on Thursday
1. Vincenzo Nibali (Italy / Astana) 73:39:35"
2. Chris Horner (U.S. / RadioShack) +3"
3. Alejandro Valverde (Spain / Movistar) +1:10"
4. Joaquim Rodriguez (Spain / Katusha) +2:24"
5. Nicolas Roche (Ireland / Saxo - Tinkoff) +3:43"
6. Domenico Pozzovivo (Italy / AG2R) +5:44"
7. Thibaut Pinot (France / FDJ.fr) +6:14"
8. Leopold Koenig (Czech Republic / NetApp) +6:35"
9. Samuel Sanchez (Spain / Euskaltel) +7:51"
10. Tanel Kangert (Estonia / Astana) +11:10"