Baxter's side comprised of several players returning from a tough period away on international duties, but he refused to blame fatigue for the loss.
"I don't think we were tired, after you lose that sounds like an excuse," the Chiefs boss said.
"I thought for whatever reason we were a bit flat in the first half but we picked it up in the second period and we got on the front foot.
"We were playing against a very difficult team and we gave them a two-goal start and you can't do that at this level."
"It was a very good pitch and we could have passed the ball better, and could have been more on the front foot in the first half," Baxter told SuperSport.
More bad news for the AmaKhosi faithful is that left-back Jimmy Jambo may miss the rest of the season after departing Friday's action on a stretcher.
Jambo appeared to have dislocated his knee after being taken out by Tinashe Nengomasha.
"Jimmy's problem is very difficult to diagnose at the moment, but he is probably out for the season," Baxter later revealed.