Sporting director Braulio Vazquez told a news conference the club had no immediate replacement for the Argentine, who took over from Unai Emery at the end of last season.
"We have been exchanging opinions, debating and we have taken the decision to sack Pellegrino," club president Manuel Llorente said.
"As you may imagine, it was a painful decision but we think that as part of the responsibility we have, we have to take these kinds of decisions," he added.
"If we are in this situation it's not just because of Mauricio Pellegrino... I think that all of us in charge of the club are responsible.
"Our aim is to get the team into the Champions League and it's our responsibility to make sure that happens. It's easier not to take decisions but it's our duty."
Pellegrino, a former Valencia player, has guided the club into the last 16 of the Champions League this season and they still have a chance to top Group F ahead of Bayern Munich depending on the results of next week's games.
However, a poor start to their domestic campaign and nine goals conceded in their last two La Liga outings was too much to take for Llorente.
Pellegrino becomes the second coach in Spain's top flight to lose his job this season following the sacking of his compatriot Mauricio Pochettino by Espanyol on Monday.
Before his dismissal, Pellegrino had told Saturday's post-match news conference he was still relishing the challenge of leading Valencia this season.
His side had to play more than half of Saturday's game with 10 men after Jonas was shown a straight red card for elbowing an opponent shortly before the break.