The 2008 Beijing Olympic time trial champion said he needed some rest before September's World Championships, at which he will be aiming for a fifth world time trial title.
"I'm going to go home and have a break. It's been a long season," he said.
"I've had 59 days of racing (this year) and I have another big objective at the end of the season with the World Championships.
"It's important that I get some rest."
Cancellara is the third Trek rider to abandon the Tour following 2010 champion Andy Schleck and Dutchman Danny van Poppel.
Apart from being a time trial specialist, Cancellara is also one of the greatest one-day racers of all time.
He has won three of the five Monument races, including Paris-Roubaix and the Tour des Flandres three times each, with one victory in Milan-San Remo.