After midfielder Pantelis Kafes leveled for Olympiakos in the 48th on a 20-meter (yard) drive substitute Roberto Soldado notched the winner for Madrid on an 87th-minute header.
In another key result among eight second-round games on Wednesday defending champion Liverpool drew 0-0 at Anfield against English champion Chelsea. Liverpool which ousted Chelsea in last year's semifinals faces Chelsea again Sunday in a league match.
Among the winners were Italy's Inter Milan Turkey's Fenerbahce Spanish club Real Betis French champion Lyon and Slovakia's Artmedia Bratislava.
In the Group F game in Madrid Raul's goal moved him one ahead of Alfredo di Stefano and three ahead of Man United striker Ruud van Nistelrooy on the all-time scoring list for Europe's No. 1 cup.
The Spanish striker headed the ball behind Greek 'keeper Antonis Nikopolidis in the ninth minute connecting on a long-range pass from David Beckham off the right wing.
In the other Group F game Lyon won 1-0 at Rosenborg on Cristiano Cris' first-half goal. Lyon which defeated Real Madrid 3-0 in the first round leads the group with six points followed by Rosenborg and Madrid with three and Olympiakos without a point.
In other results - Group E: Fenerbahce 3 PSV Eindhoven 0; Schalke 2 AC Milan 2. Group G: Anderlecht 0 Real Betis 1; Liverpool 0 Chelsea 0. Group H: Porto 2 Artmedia Bratislava 3; Inter Milan 1 Rangers 0.
In Group E last season's runner-up AC Milan stayed top despite the 2-2 draw at Schalke. Milan has four points followed by Fenerbahce and PSV with three and Schalke with one.
AC Milan led twice with goals from Clarence Seedorf and Andriy Shevchenko. Soren Larsen leveled in the third minute after Seedorf's first-minute goal and Hamit Altintop equalized again in the 70th.
Fenerbahce picked up its 3-0 home win over PSV on two goals from Alex de Souza and a late goal from Stephen Appiah.
In Group G Liverpool and Chelsea - the group favorites - moved up to four points. Spain's Betis playing the first time in the Champions League improved to three points with Anderlecht having lost its first two.
"I'm not 100 percent (satisfied) but it's a point so I'm not crazy with happiness but I'm not very sad Chelsea manager Jose Mourinho said. "A point in the Champions League away from home is a positive result."
Liverpool manager Rafa Benitez was upset that two possible penalties were not awarded.
"The positive thing is that we have a point he said. "But we lost today two points because for me we played better than them."
Brazilian striker Oliveira scored for Betis in the 68th. After collecting a pass through the center from playmaker Joaquin he fired a low shot into the corner. It was Anderlecht's ninth straight Champions League loss.
In Group H Inter Milan leads with two straight victories. David Pizarro scored the winner in the 49th to beat Rangers. The game was played in an empty San Siro stadium part of a fan-ban after flares were thrown on the field in last year's Champions League quarterfinal against AC Milan - injuring AC Milan goalkeeper Dida.
Pizarro's first Champions League goal came as he started in place of ineligible Juan Sebastian Veron.
"Whenever the team calls on me I give my all Pizarro said.
Rangers and Artmedia Bratislava have three points with 2004 European champion Porto without a point.
Playing its first time in the Champions League Balazs Borbely scored the winner in the 74th to break a 2-2 tie for the Bratislava team. Luis Gonzalez and Diego Ribas put Porto ahead 2-0 after 39 minutes but Peter Petras and Jan Kozak evened the score after 54 minutes.
"If we wanted to finish in the top two we had to win today Porto's Dutch coach Co Adriaanse said. "It'll be very difficult now."
In Tuesday second-round games:
Group A: Bayern Munich 1 FC Brugge 0; Juventus 3 Rapid Vienna 0. Group B: Ajax 1 Arsenal 2; FC Thun 1 Sparta Prague 0. Group C: Barcelona 4 Udinese 1; Panathinaikos 2 Werder Bremen 1. Group D: Lille 0 Villarreal 0; Manchester United 2 Benfica 1.