The Chiefs, riding high after a win over the Crusaders last week, were heavy favourites going into the match against a team that finished 13th last year, but found themselves in a dogfight against the Canberra team.
The match only sparked to life in the final 15 minutes, and the Chiefs scored two tries to one in what was a frantic but mistake-ridden game.
"We were going backward a bit of the time but showed a fair bit of character," Chiefs captain Craig Clarke said.
"They were in up our faces and we were struggling to get go-forward and their defence was good, so it was fairly testing."
The Chiefs showed a willingness to throw the ball around early, almost going ahead after a break from Sonny Bill Williams but Rugby World Cup-winning coach Jake White's Brumbies kept them out with committed defence.
The teams traded two penalties apiece in a high-tempo opening 20 minutes, the Brumbies matching their opponents' attacking intent but letting themselves down with handling errors.
Flyhalf Aaron Cruden put the Chiefs ahead with another penalty before Scott Fardy came close to a try for the Brumbies in the 28th minute, only to knock on as he stretched to ground the ball after a desperate tackle from Tim Kerr-Barlow.
The Brumbies did not allow the Chiefs' forward pack, which laid the foundations for wins over the Crusaders and Auckland Blues, to dominate, forcing the New Zealanders to fruitlessly search for space out wide.
Christian Lealiifano kicked home penalties either side of half-time to give the Brumbies a narrow lead but the arm-wrestle continued when Cruden slotted his fourth to make it 12-12.
The match looked like it was going to be settled by penalties as Cruden and Lealiifano again exchanged three-pointers before Chiefs prop Sona Taumalolo finally crossed the line for the Chiefs's first try after 68 minutes.
It came after Liam Messam was held up in the in-goal area, resulting in a five-metre scrum that allowed the Chiefs' forwards to turn the screws on the Brumbies defence and allow Taumalolo to bulldoze his way in for the score.
The Brumbies hit back in the 75th minute when Sam Carter touched down out wide, setting up a blockbuster finale with scores again deadlocked at 22-all.
Jackson
Willison provided the match-winning try in the 79th minute, scrambling
over in the corner, with Cruden's conversion making it 29-22.
Teams:
15. Robbie Robinson, 14. Richard Kahui, 13. Jackson Willison, 12. Sonny Bill Williams, 11. Tim Nanai-Williams, 10. Aaron Cruden, 9. Tawera Kerr-Barlow, 8. Liam Messam, 7. Sam Cane, 6. Tanerau Latimer, 5. Brodie Retallick, 4. Craig Clarke (captain), 3. Ben Tameifuna, 2. Mahonri Schwalger, 1. Sona Taumalolo
Substitutes: 16. Marcel Cummings-Toone, 17. Josh Hohneck, 18. Kane Thompson, 19. Scott Waldrom, 20. Augustine Pulu, 21. Martino Nemani / Andrew Horrell
Brumbies:
15 Jesse Mogg, 14 Pat McCabe, 13 Andrew Smith, 12 Joe Tomane, 11 Henry Speight, 10 Christian Lealiifano, 9 Ian Prior, 8 Ben Mowen, 7 Michael Hooper, 6 Scott Fardy, 5 Peter Kimlin, 4 Leon Power, 3 Dan Palmer, 2 Stephen Moore, 1 Ben Alexander
Substitutes: 16 Anthony Hegarty, 17 Ruaidhri Murphy, 18 Sam Carter, 19 Ita Vaea, 20 Nic White, 21 Zack Holmes, 22 Cam Crawford