Hennie Daniller's 23rd minute score and eight points from the boot of Johan Goosen had given the visitors a 13-3 half-time lead after an error-strewn performance from the defending champions in the opening 40 minutes.
The Sharks came out a reinvigorated team after the break, scoring 17 unanswered points to seal a final meeting with either the Lions or Western Province, who meet in Johannesburg later on Saturday.
Victory looked unlikely following a poor opening by John Plumtree's men.
After a after a cagey opening, in which both made mistakes, the Sharks should have been ahead in the eighth minute, but Patrick Lambie seemed to rush his penalty from straight in front as an early chance was missed.
Nerves continued to show up, especially from the home side, who then conceded a penalty for a high tackle, which was calmly slotted over by Goosen.
Frederic Michalak then fluffed another easy chance at goal when the Cheetahs were caught off-side - the Frenchman looked completely out of rhythm - before a mammoth penalty from inside his own half close to the right touchline by Goosen extended the lead.
The Cheetahs continued to dominate with their exuberant number 10 making a fine break through the middle to set up a strong attacking phase just after the 20 minute mark.
Stefan Terblanche managed to clear to the 22, but the onslaught continued, and after a quick tap-and-go penalty by Adriaan Strauss, the ball was spread wide to the left by the captain where Daniller strolled over with the home side caught completely flat-footed.
Goosen extended the lead to 13-0 with the conversion, before Michalak missed another straightforward shot at goal shortly before being forced off with what appeared to be a calf problem.
In the 32nd minute, the Sharks got closer when Willem Alberts knocked on with the try-line beckoning, a decision that was made by television official Jerome Fortuin.
However, they were finally on the board five minutes before the break when Lambie slotted over from the right, much to the relief of the Kings Park crowd.
Goosen strangely missed a kick at goal, albeit another near 50-metre one, and a drop-goal attempt before half-time, but the men from Bloemfontein went into the break 10 points ahead.
A strong message during the interval from Plumtree seemed to have reinvigorated his team.
Four minutes after the restart, Mtawarira was in thanks to a strong start to the second period - the Springbok prop picking up strongly from the ruck to power over.
Lambie added the extras and ten minutes later the hosts were in front when an attack was quickly switched from right to left with Ndungane unmarked as he ran in under the post.
South Africa Player of the Year nominee, Lambie, again made no mistake with the conversion, but he should have taken the gap to seven points just before the hour mark when he wasted another penalty.
He made amends five minutes from the end slotting over another kick that put the Sharks on course to defend their title.
Sharks:
15. Patrick Lambie, 14. Odwa Ndungane, 13. Stefan Terblanche, 12. Marius Joubert, 11. JP Pietersen, 10. Frederic Michalak, 9. Conrad Hoffmann, 8. Ryan Kankowski, 7. Willem Alberts, 6. Keegan Daniel (captain), 5. Ross Skeate, 4. Jean Deysel/Anton Bresler, 3. Jannie du Plessis, 2. Bismarck du Plessis, 1. Tendai "Beast" Mtawarira
Substitutes: 16. Craig Burden, 17. Eugene van Staden, 18. Marcell Coetzee, 19. Jacques Botes, 20. Ross Cronje, 21. Adrian Jacobs, 22. Lwazi Mvovo
Free State Cheetahs:
15 Hennie Daniller, 14 Ryno Benjamin, 13 Robert Ebersohn, 12 Andries Strauss, 11 Cameron Jacobs, 10 Johan Goosen, 9 Piet van Zyl, 8 Boom Prinsloo, 7 Ashley Johnson, 6 Pieter Labuschagne, 5 Izak van der Westhuizen, 4 Philip van der Walt, 3 WP Nel, 2 Adriaan Strauss (captain), 1 Marcel van der Merwe
Substitutes: 16 Hercu Liebenberg, 17 Trevor Nyakane, 18 Wilhelm Steenkamp, 19 Johan Wessels, 20 Tewis de Bruyn, 21 Sias Ebersohn, 22 Nico Scheepers