The 29-year-old Harvey, on a short-term contract as the Force struggle to cope with the many injuries to their squad, scored a try and kicked three penalties and a conversion to end his new side's run of three losses.
It was a fully deserved win for the Force, who controlled possession and territory for long periods and bossed the breakdown against a lethargic Waratahs side.
"Very proud of the team and the way we gutsed it out," Force captain and openside flank David Pocock said in a pitchside interview.
"It's been a tough time as a group but we know where we are going and we have hung pretty tough and we have got our first win."
The home side started brightly with a try in the third minute, Wallabies utility back Adam Ashley-Cooper touching down after loitering at the back of a powerful Waratahs rolling maul.
Harvey slotted over a 10th-minute penalty to cut the deficit and the momentum swung fully in the Force's favour in the 16th minute when Waratahs lock Dean Mumm was sent to the sin bin.
A minute after the lock was penalised for hitting the ball out of the hands of Force scrumhalf Brett Sheehan, Harvey crossed for a try after cutting a great line and collecting a nifty offload from flyhalf James Stannard.
The Force dominated possession from then on but conceded a second try just before halftime when Waratahs winger Tom Kingston collected a loose ball and won a sprint to the corner.
Television replays showed the try should not have been awarded with Waratahs scrumhalf Sarel Pretorius, whose kick Kingston collected, knocking the ball on when he forced the turnover with a tackle.
Flyhalf Daniel Halangahu converted brilliantly from the touchline but another Harvey penalty cut the Waratahs lead to 14-13 at halftime despite the home side having only 25 percent of the ball.
Halangahu kicked two penalties and Harvey one at the start of the second half before the Force scored the game-winning try through Samu Wara.
Force lock Nathan Sharpe deftly flicked a pass to hooker Nathan Charles and he fed Wara with the winger cutting inside and touching down to give the visitors a 21-20 lead in the 64th minute.
"We let ourselves down and that is what is going to happen in Super Rugby if you don't turn up on your game," Halangahu said after the Waratahs' second loss in four matches.
Teams:Waratahs:
15 Bernard Foley, 14 Tom Kingston, 13 Rob Horne, 12 Berrick Barnes, 11 Adam Ashley-Cooper, 10 Daniel Halangahu (captain), 9 Sarel Pretorius, 8 Tevita Metuisela, 7 Jono Jenkins, 6 Dave Dennis, 5 Kane Douglas, 4 Dean Mumm, 3 Sekope Kepu, 2 Tatafu Polota-Nau, 1 Benn Robinson
Substitutes: 16 John Ulugia, 17 Paddy Ryan, 18 Sitaleki Timani, 19 Lopeti Timani, 20 Chris Alcock, 21 Brendan McKibbin, 22 Tom Carter
Force:
15. David Harvey, 14. Samu Wera, 13. Will Tupou, 12. Rory Sidey, 11. Alifeleti Mafi, 10. James Stannard, 9. Brett Sheehan, 8. Matthew Hodgson, 7. David Pocock (captain), 6. Angus Cottrell, 5. Nathan Sharpe, 4. Sam Wykes, 3. Salesi Ma'afu, 2. Nathan Charles, 1. Pekahou Cowan
Substitutes: 16. Ben Whittaker, 17. Kieran Longbottom, 18. Phoenix Battye, 19. Richard Brown, 20. Justin Turner, 21. Ben Seymour, 22. Winston Stanley