The 27-year-old Colombian midfielder arrived on loan in 2017 from Real Madrid and Bayern have the option to buy him.
"The decision was taken because he came to see me before the end of the season to ask me not to take up the option," Rummenigge, Bayern's chairman, told German media.
"Exercising the option against his will would have made no sense," said Rummenigge. "You shouldn't sign a player for 42 million euros, with a corresponding salary, if you can't offer him a starting place on the team."
"Here he has no guarantee," said Rummenigge. "Personally, I regret it".
Rummenigge, a former German international striker, said he was a "fan" adding that James "is a player of great qualities."
In 67 matches with Munich, James scored 15 goals and provided 20 assists. Sometimes brilliant, he has also often been out of favour this season under Niko Kovac.
He appears to have no future at Real Madrid. He has been linked with a move to Napoli, where he would be reunited with coach Carlo Ancelotti, who was manager when James arrived at Bayern in July 2017, but was sacked before the end of September.
Rummenigge ruled out buying James just to make an immediate profit selling him.
"It wouldn't be fair to Real and it's not the kind of thing FC Bayern do. We do not traffic human beings."