As a result the Amakhosi climbed up to fourth on the league table, while Rise and Shine dropped down to position five on the standings.
Amakhosi enjoyed most of the possession in the early stages of the game, but things changed when Willard Katsande committed a penalty in the 22nd minute fouling Puleng Tlolane in the box.
Tlolane's Polokwane teammate Rodney Ramagalela then stepped up, but his effort from the spot-kick was well-saved by Chiefs goalkeeper Brilliant Khuzwayo, who guessed the right way.
But Khuzwayo was beaten in the 30th minute and it was Ramagalela, who drilled home a fine effort from 20-yards out to make it 1-0 to Polokwane - silencing the Chiefs fans in the process.
Chiefs did press for the equalising goal before the half-time break. However, Polokwane stood firm at the back until the referee blew the half-time whistle.
Amakhosi began the second-half lively and they managed to restore parity through Bernard Parker, who netted his first league goal of the season, two minutes after the hour-mark.
In the 71st minute, the Soweto giants took the lead for the first time when their new signing Joseph Molangoane made it 2-1 with a good finish - netting his first goal for the club in the process.
Rise and Shine managed to level matters with eight minutes left on the clock thanks to hard-working forward, Tlolane.
However, Zimbabwe captain Katsande snatched the winning goal for Chiefs in stoppage, much to the delight of the home crowd as the Amakhosi ended their seven-match winless streak.
Teams:
Kaizer Chiefs
Khuzwayo, Mphahlele (Moleko 57'), Khumalo, Gordinho, Mathoho, Katsande, Maluleka (Ekstein 70'), Molangoane, Tshabalala, Parker, Macha (Letlotlo 45').
Polokwane City
Chigova, Maponyane (Jembula 77'), Semenya, Matsemela, Tema, Maluleke, Klaas, Seabi (Mbonani 45'), Ramagalela, Tlolane, Mncwango (Ndou 90')