Lehlohonolo Majoro once again came to Amakhosi's rescue as he fired home seven minutes into added-time to hand his side three invaluable points after Lerato Chabangu and Bennett Chenene both scored either side of Reneilwe Letsholonyane's brace.
The result ttook Chiefs seven points clear at the top of the PSL standings, although Baxter admits his side made it harder for themselves than it needed to be.
"It is a good win for us and I have to congratulate the players for a job well done," Baxter told his club's official website.
"The win though was not good for the hearts of people who were sitting on the bench.
"I was once again pleased in the way we started the game. We always try to play at high tempo and we created quite a number of chances some I think we should have scored.
"Once again we allowed Moroka Swallows to come back into the game. We were aware of the quality in their side and we knew that when they are on top gear they are capable of beating anyone in the league.
"I was not happy that we allowed them to come back twice. There were some elements of the performance that I will have to address with the team. We needed to keep things a bit tight at the back and we have to sort this out.
He continued: "We changed things around in the second half introducing Siyabonga (Nkosi) to play just behind the strikers and then we threw in Kingston (Nkhatha). At the end it worked for us."