"What tennis is becoming scares me," said Rosset, who reached a career high world number nine in 1995 and has 15 ATP titles to his name.
Rosset, now a pundit with a Swiss TV channel, added: "Everyone talks only about 'the Big Four, the Big Four'. It shows that tennis rests on only four players."
Since 2008, the same four players - Novak Djokovic, Roger Federer, Rafael Nadal and Andy Murray - have finished the season in the top four.
Rosset believes the problem stems from the fact that in the grand slam tournaments the number of seeded players has been doubled to 32 since Wimbledon 2001 while in the Masters events 16 are seeded instead of eight previously.
"The priority of tournament directors is to protect them (the seeded players) so they have a good match at the weekend.
"There used to be 10, 15, 20 star players. Now you only have four. Fifteen years ago, every player in the top 10 had won a grand slam or a Masters."
Rosset said that first-week action in the grand slams was often too dull.
"With 32 seeded players, there is no big match. You can't have Rafael Nadal v John Isner in the first round of Wimbledon," he said.
"We used to say the women's tournaments would start in the quarter-finals. It's becoming the same with men's tournaments.
"One year, when I was world number 25, I played Andre Agassi in the first round at Roland Garros, Stefan Edberg at Wimbledon and (the following year) Goran Ivanisevic at the U.S. Open.
"This can't happen anymore. In these conditions, it's tough for the young players to oust the old players."
Rosset believes the standardisation of surfaces is also part of the problem.
"If you take a final at Roland Garros or Wimbledon, it's almost the same. Grass got slower as clay got quicker - so you always see the same players.
"When you see Tommy Haas in the top 10 at 35, it's good for him but what does that mean? Well before that age, (Pete) Sampras and Agassi were out already. The young players had taken over."
Had tennis officials decided to have clearly different surfaces, said Rosset, the list of honours would not be quite the same.
"Wimbledon is not played on clay, sure, but Sampras would not win seven titles on today's surface and Nadal would not win Wimbledon on the previous grass," he added.
"Agassi who beat Ivanisevic at Wimbledon (1992) was brilliant but since then they slowed the surface down. What you get is a Lleyton Hewitt v David Nalbandian (2002) with no player going to the net once."