Bergkamp scored his first Champions League goal in three years to help the Gunners to a 2-1 victory at Highbury.
Wesley Sneijder scored a last-minute equalizer to earn four-time champion Ajax a 1-1 draw at Sparta Prague and Italian striker Fabrizio Miccoli netted an injury-time winner for Benfica in a 1-0 victory over Lille.
Also Vincenzo Iaquinta scored a hat trick as Udinese marked its debut in the competition with a 3-0 victory over Panathinaikos and Serie A champion Juventus won 2-1 at FC Brugge despite having Patrick Vieira sent off a minute from the end.
Other powerhouse teams started with victories. German champion Bayern Munich won 1-0 at Rapid Vienna and FC Barcelona topped host Werder Bremen 2-0. Manchester United however was held 0-0 at Villarreal and had to play the final 26 minutes without Wayne Rooney who was ejected for ironically applauding referee Kim Milton Nielsen after a caution.
Rooney whose temper has started to boil in recent games for England had little support from United manager Alex Ferguson.
"You can't applaud the referee Ferguson said. "He's a young man and didn't give himself a chance. He's a fiery character and when he's thinking he's been done an injustice he reacts like that."
At Highbury Nelson Ferreira leveled for Thun two minutes after Gilberto Silva scored for Arsenal which saw Robin van Persie sent off for a dangerously high kick in first-half injury time.
A 1-1 draw looked likely as the game went into the final minute but Bergkamp dug the ball out of a scramble and scored the winner.
"We created some chances in the first half and the sending-off made it difficult for us to get going Bergkamp said. "But we did and we got a goal. They equalized but we kept fighting and there was a good spirit. Maybe in the end we would have been happy with a point with it being 10 against 11."
With Sparta Prague and Ajax drawing Bergkamp's goal put the Gunners at the top of Group B after the first round of games.
Pavel Nedved and David Trezeguet scored within nine minutes of each other as Juventus won 2-1 at FC Brugge.
"The two yellows were very severe said Vieira after his first expulsion in a Juve shirt. "What was important was the three points."
Peru striker Paolo Guerrero scored Bayern's winner in Vienna to leave the champions of Italy and Germany atop Group A.
Iaquinta recently called into the Italian national squad scored all three for Udinese against Panathinaikos netting his first in the 28th and then adding two more in a three-minute spell in the second half.
The result put Udinese above Barcelona on goal difference although goals by Deco and Ronaldinho gave the Spanish league champion a 2-0 triumph at Werder Bremen.
Miccoli's late strike for Benfica was the only goal in Group D although he missed other chances in a 1-0 victory over Lille.
Rooney whose temper flared during England's 1-0 loss to Northern Ireland a week ago was sent off in the 64th for two yellow cards at Villarreal. The second was for ironic applause of referee Kim Milton Nielsen's decision to caution him for a foul moments after getting his first card.
Villarreal was weakened by the absence of injured Argentine playmaker Juan Roman Riquelme.
"We weren't too sharp up front. But a draw's not a bad result said Villarreal striker Diego Forlan who was playing against one of his former clubs.
The highlight result of Tuesday's games was Lyon's 3-0 victory over nine-time champion Real Madrid in Group F. Norway's Rosenborg won 3-1 at Olympiakos in the same group.
Defending champion Liverpool won 2-1 at Real Betis in Group G while Chelsea beat Anderlecht 1-0. Also it was: AC Milan 3 Fenerbahce 1 and PSV Eindhoven 1 Schalke 0 in Group E; Artmedia Bratislava 0 Inter Milan 1 and Rangers 3 FC Porto 2 in Group H