The Indian defence kept the Asian champions at bay for the first 44 minutes before former Boca Juniors star Takayuki Suzuki put Japan ahead on the stroke of half-time in front of 95000 fans at the Salt Lake stadium.
Suzuki's strike opened the floodgates as Japan pumped in three more goals after the break through Shinji Ono Takashi Fukunishi and Tsuneyasu Miyamoto.
It was the third defeat in four matches for the Indians who had been trounced 7-0 by Japan in the first leg at Saitama in June.
Japan lead the group with 12 points from four games while main rivals Oman have nine points following their 2-0 win over Singapore earlier on Wednesday.
The newly crowned Asian Cup holders need only a point from their two remaining games against Oman and Singapore to go through to the next round.
"We came here to win and got that said the Brazilian legend. "It took the boys time to settle down in the hot weather but they showed good courage to come back strongly in the second half."
India's English coach Stephen Constantine was happy just to have kept the score down.
"Japan showed they were the stronger side but I am proud of the way my team played especially in the first half he said.
"The defence stuck well to their task. Maybe we conceded two extra goals but that was because of inexperience."
Constantine had tried to motivate his players by showing them "Miracle" a movie about an amateur American ice hockey team which upsets a heavyweight team from Russia the night before the match.
"I believe in miracles said Constantine. "That did not happen today but there is nothing to be ashamed of."
The start of the second half was delayed by 10 minutes due to power failure in one of the floodlight towers."