Federer had to pull out all stops to get past the unseeded Simon, winning 6-2, 6-3, 4-6, 4-6, 6-3 in 3hr 13min on Rod Laver Arena.
The Swiss world number two had cruised to a two-set advantage, but Simon, who had won their only other two encounters, hit back spectacularly to take the contest into a fifth and deciding set.
Federer was relieved to escape with a five-set victory, and prevent losing for the first time in the second round at a Grand Slam event since the 2003 French Open.
"Today I got lucky to go through," Federer said.
"He's a great player. Matches against him don't come easy, and they always go the distance. Hopefully I won't play him any more," he laughed.
Federer will next play Belgium's Xavier Malisse in Friday's third round.
Federer's victory equalled Swede Stefan Edberg's record (56-10) for the most Open Era wins at the Australian Open. Federer now has a 56-7 win-loss record at Melbourne Park.
The all-time Grand Slam champion, with 16 major titles, is bidding to become the second man to win five Australian titles behind six-time winner Roy Emerson.
Collated results from day three of the Australian Open (x denotes seeding):
Men's singles
Tomas Berdych (CZE x6) bt Philipp Kohlschreiber (GER) 4-6, 6-2, 6-3, 6-4Richard Gasquet (FRA x28) bt Adrian Mannarino (FRA) 6-3, 7-6 (7/4), 6-4
Kei Nishikori (JPN) bt Florian Mayer (GER) 6-4, 6-3, 0-6, 6-3
Fernando Verdasco (ESP x9) bt Janko Tipsarevic (SRB) 2-6, 4-6, 6-4, 7-6 (7/0), 6-0
Nicolas Almagro (ESP x14) bt Igor Andreev (RUS) 7-5, 2-6, 4-6, 7-6 (12/10), 7-5
Ivan Ljubicic (CRO x17) bt Benoit Paire (FRA) 6-3, 6-7 (2/7), 6-4, 7-6 (7/5)
Viktor Troicki (SRB x29) bt Nicolas Mahut (FRA) 6-4, 6-2, 1-6, 6-3
Andy Roddick (USA x8) bt Igor Kunitsyn (RUS) 7-6 (9/7), 6-2, 6-3
Robin Haase (NED) bt Juan Monaco (ARG x26) 6-4, 6-4, 3-6, 6-2
Tommy Robredo (ESP) bt Mardy Fish (USA x16) 1-6, 6-3, 6-3, 6-3
Sergiy Stakhovsky (UKR) bt Lukasz Kubot (POL) 6-3, 6-4, 6-4
Xavier Malisse (BEL) bt Albert Montanes (ESP x25) 6-4, 6-0, 6-1
Novak Djokovic (SRB x3) bt Ivan Dodig (CRO) 7-5, 6-7 (8/10), 6-0, 6-2
Stanislas Wawrinka (SUI x19) bt Grigor Dimitrov (BUL) 7-5, 6-3, 6-3
Gael Monfils (FRA x12) bt Frederico Gil (POR) 6-4, 6-3 1-6, 6-2
Roger Federer (SUI x2) bt Gilles Simon (FRA) 6-2, 6-3, 4-6, 4-6, 6-3
Women's singles
Caroline Wozniacki (DEN x1) bt Vania King (USA) 6-1, 6-0
Dominika Cibulkova (SVK x29) bt Alberta Brianti (ITA) 6-1, 4-6, 6-2
Anastasija Sevastova (LAT) bt Yanina Wickmayer (BEL x21) 6-4, 6-2
Vesna Manasieva (RUS) bt Marion Bartoli (FRA x15) 3-6, 6-3, 6-0
Justine Henin (BEL x11) bt Elena Baltacha (GBR) 6-1, 6-3
Svetlana Kuznetsova (RUS x23) bt Arantxa Rus (NED) 6-1, 6-4
Monica Niculescu (ROM) bt Tsvetana Pironkova (BUL x32) 6-4, 6-1
Francesca Schiavone (ITA x6) bt Rebecca Marino (CAN) 6-3, 5-7, 9-7
Venus Williams (USA x4) bt Sandra Zahlavova (CZE) 6-7 (6/8), 6-0, 6-4
Julia Goerges (GER) bt Kaia Kanepi (EST x20) 6-4, 3-6, 6-4
Maria Sharapova (RUS x14) bt Virginie Razzano (FRA) 7-6 (7/3), 6-3
Chanelle Scheepers (RSA) bt Regina Kulikova (RUS) 6-4, 4-6, 7-5
Victoria Azarenka (BLR x8) bt Andrea Hlavackova (CZE) 6-4, 6-4
Barbora Zahlavova Strycova (CZE) bt Jelena Dokic (AUS)
Li Na (CHN) bt Evgeniya Rodina (RUS) 6-3, 6-2
Andrea Petkovic (GER x30) bt Anne Keothavong (GBR) 2-6, 7-5, 6-0